3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1630 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 517
Associated Job Ids:
1631
DAG Visualization
digraph G { subgraph clusterstage_1630 { label="Stage 1630"; 17120 [labelType="html" label="SQLExecutionRDD [17120]<br>run at <unknown>:0"]; 17134 [labelType="html" label="SQLExecutionRDD [17134]<br>toJavaRDD at <unknown>:0"]; 17136 [labelType="html" label="PythonRDD [17136]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster38306 { label="MicroBatchScan"; 17116 [labelType="html" label="DataSourceRDD [17116]<br>run at <unknown>:0"]; 17117 [labelType="html" label="MapPartitionsRDD [17117]<br>run at <unknown>:0"]; } subgraph cluster38305 { label="Project"; 17118 [labelType="html" label="MapPartitionsRDD [17118]<br>run at <unknown>:0"]; } subgraph cluster38301 { label="WholeStageCodegen (1)"; 17119 [labelType="html" label="MapPartitionsRDD [17119]<br>run at <unknown>:0"]; } subgraph cluster38323 { label="WholeStageCodegen (1)"; 17127 [labelType="html" label="MapPartitionsRDD [17127]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38322 { label="Filter"; 17128 [labelType="html" label="MapPartitionsRDD [17128]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38321 { label="Project"; 17129 [labelType="html" label="MapPartitionsRDD [17129]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38318 { label="WholeStageCodegen (2)"; 17130 [labelType="html" label="MapPartitionsRDD [17130]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38317 { label="Project"; 17131 [labelType="html" label="MapPartitionsRDD [17131]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38316 { label="DeserializeToObject"; 17132 [labelType="html" label="MapPartitionsRDD [17132]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38315 { label="MapPartitions"; 17133 [labelType="html" label="MapPartitionsRDD [17133]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38325 { label="mapPartitions"; 17135 [labelType="html" label="MapPartitionsRDD [17135]<br>toJavaRDD at <unknown>:0"]; } } 17116->17117; 17117->17118; 17118->17119; 17119->17120; 17120->17127; 17127->17128; 17128->17129; 17129->17130; 17130->17131; 17131->17132; 17132->17133; 17133->17134; 17134->17135; 17135->17136; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time