3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1167 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 476
Associated Job Ids:
1168
DAG Visualization
digraph G { subgraph clusterstage_1167 { label="Stage 1167"; 12269 [labelType="html" label="SQLExecutionRDD [12269]<br>run at <unknown>:0"]; 12277 [labelType="html" label="SQLExecutionRDD [12277]<br>toJavaRDD at <unknown>:0"]; 12279 [labelType="html" label="PythonRDD [12279]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster27449 { label="MicroBatchScan"; 12265 [labelType="html" label="DataSourceRDD [12265]<br>run at <unknown>:0"]; 12266 [labelType="html" label="MapPartitionsRDD [12266]<br>run at <unknown>:0"]; } subgraph cluster27448 { label="Project"; 12267 [labelType="html" label="MapPartitionsRDD [12267]<br>run at <unknown>:0"]; } subgraph cluster27444 { label="WholeStageCodegen (1)"; 12268 [labelType="html" label="MapPartitionsRDD [12268]<br>run at <unknown>:0"]; } subgraph cluster27458 { label="WholeStageCodegen (1)"; 12270 [labelType="html" label="MapPartitionsRDD [12270]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27457 { label="Filter"; 12271 [labelType="html" label="MapPartitionsRDD [12271]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27456 { label="Project"; 12272 [labelType="html" label="MapPartitionsRDD [12272]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27453 { label="WholeStageCodegen (2)"; 12273 [labelType="html" label="MapPartitionsRDD [12273]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27452 { label="Project"; 12274 [labelType="html" label="MapPartitionsRDD [12274]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27451 { label="DeserializeToObject"; 12275 [labelType="html" label="MapPartitionsRDD [12275]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27450 { label="MapPartitions"; 12276 [labelType="html" label="MapPartitionsRDD [12276]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27460 { label="mapPartitions"; 12278 [labelType="html" label="MapPartitionsRDD [12278]<br>toJavaRDD at <unknown>:0"]; } } 12265->12266; 12266->12267; 12267->12268; 12268->12269; 12269->12270; 12270->12271; 12271->12272; 12272->12273; 12273->12274; 12274->12275; 12275->12276; 12276->12277; 12277->12278; 12278->12279; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time