3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1822 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 461
Associated Job Ids:
1822
DAG Visualization
digraph G { subgraph clusterstage_1822 { label="Stage 1822"; 19135 [labelType="html" label="SQLExecutionRDD [19135]<br>run at <unknown>:0"]; 19143 [labelType="html" label="SQLExecutionRDD [19143]<br>toJavaRDD at <unknown>:0"]; 19146 [labelType="html" label="PythonRDD [19146]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster42822 { label="MicroBatchScan"; 19131 [labelType="html" label="DataSourceRDD [19131]<br>run at <unknown>:0"]; 19132 [labelType="html" label="MapPartitionsRDD [19132]<br>run at <unknown>:0"]; } subgraph cluster42821 { label="Project"; 19133 [labelType="html" label="MapPartitionsRDD [19133]<br>run at <unknown>:0"]; } subgraph cluster42817 { label="WholeStageCodegen (1)"; 19134 [labelType="html" label="MapPartitionsRDD [19134]<br>run at <unknown>:0"]; } subgraph cluster42831 { label="WholeStageCodegen (1)"; 19136 [labelType="html" label="MapPartitionsRDD [19136]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42830 { label="Filter"; 19137 [labelType="html" label="MapPartitionsRDD [19137]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42829 { label="Project"; 19138 [labelType="html" label="MapPartitionsRDD [19138]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42826 { label="WholeStageCodegen (2)"; 19139 [labelType="html" label="MapPartitionsRDD [19139]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42825 { label="Project"; 19140 [labelType="html" label="MapPartitionsRDD [19140]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42824 { label="DeserializeToObject"; 19141 [labelType="html" label="MapPartitionsRDD [19141]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42823 { label="MapPartitions"; 19142 [labelType="html" label="MapPartitionsRDD [19142]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42833 { label="mapPartitions"; 19144 [labelType="html" label="MapPartitionsRDD [19144]<br>toJavaRDD at <unknown>:0"]; } } 19131->19132; 19132->19133; 19133->19134; 19134->19135; 19135->19136; 19136->19137; 19137->19138; 19138->19139; 19139->19140; 19140->19141; 19141->19142; 19142->19143; 19143->19144; 19144->19146; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time