3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1584 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 401
Associated Job Ids:
1585
DAG Visualization
digraph G { subgraph clusterstage_1584 { label="Stage 1584"; 16643 [labelType="html" label="SQLExecutionRDD [16643]<br>run at <unknown>:0"]; 16651 [labelType="html" label="SQLExecutionRDD [16651]<br>toJavaRDD at <unknown>:0"]; 16653 [labelType="html" label="PythonRDD [16653]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster37233 { label="MicroBatchScan"; 16639 [labelType="html" label="DataSourceRDD [16639]<br>run at <unknown>:0"]; 16640 [labelType="html" label="MapPartitionsRDD [16640]<br>run at <unknown>:0"]; } subgraph cluster37232 { label="Project"; 16641 [labelType="html" label="MapPartitionsRDD [16641]<br>run at <unknown>:0"]; } subgraph cluster37226 { label="WholeStageCodegen (1)"; 16642 [labelType="html" label="MapPartitionsRDD [16642]<br>run at <unknown>:0"]; } subgraph cluster37242 { label="WholeStageCodegen (1)"; 16644 [labelType="html" label="MapPartitionsRDD [16644]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37241 { label="Filter"; 16645 [labelType="html" label="MapPartitionsRDD [16645]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37240 { label="Project"; 16646 [labelType="html" label="MapPartitionsRDD [16646]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37237 { label="WholeStageCodegen (2)"; 16647 [labelType="html" label="MapPartitionsRDD [16647]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37236 { label="Project"; 16648 [labelType="html" label="MapPartitionsRDD [16648]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37235 { label="DeserializeToObject"; 16649 [labelType="html" label="MapPartitionsRDD [16649]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37234 { label="MapPartitions"; 16650 [labelType="html" label="MapPartitionsRDD [16650]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37244 { label="mapPartitions"; 16652 [labelType="html" label="MapPartitionsRDD [16652]<br>toJavaRDD at <unknown>:0"]; } } 16639->16640; 16640->16641; 16641->16642; 16642->16643; 16643->16644; 16644->16645; 16645->16646; 16646->16647; 16647->16648; 16648->16649; 16649->16650; 16650->16651; 16651->16652; 16652->16653; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time