3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1157 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 449
Associated Job Ids:
1158
DAG Visualization
digraph G { subgraph clusterstage_1157 { label="Stage 1157"; 12164 [labelType="html" label="SQLExecutionRDD [12164]<br>run at <unknown>:0"]; 12172 [labelType="html" label="SQLExecutionRDD [12172]<br>toJavaRDD at <unknown>:0"]; 12174 [labelType="html" label="PythonRDD [12174]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster27214 { label="MicroBatchScan"; 12160 [labelType="html" label="DataSourceRDD [12160]<br>run at <unknown>:0"]; 12161 [labelType="html" label="MapPartitionsRDD [12161]<br>run at <unknown>:0"]; } subgraph cluster27213 { label="Project"; 12162 [labelType="html" label="MapPartitionsRDD [12162]<br>run at <unknown>:0"]; } subgraph cluster27209 { label="WholeStageCodegen (1)"; 12163 [labelType="html" label="MapPartitionsRDD [12163]<br>run at <unknown>:0"]; } subgraph cluster27223 { label="WholeStageCodegen (1)"; 12165 [labelType="html" label="MapPartitionsRDD [12165]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27222 { label="Filter"; 12166 [labelType="html" label="MapPartitionsRDD [12166]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27221 { label="Project"; 12167 [labelType="html" label="MapPartitionsRDD [12167]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27218 { label="WholeStageCodegen (2)"; 12168 [labelType="html" label="MapPartitionsRDD [12168]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27217 { label="Project"; 12169 [labelType="html" label="MapPartitionsRDD [12169]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27216 { label="DeserializeToObject"; 12170 [labelType="html" label="MapPartitionsRDD [12170]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27215 { label="MapPartitions"; 12171 [labelType="html" label="MapPartitionsRDD [12171]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27225 { label="mapPartitions"; 12173 [labelType="html" label="MapPartitionsRDD [12173]<br>toJavaRDD at <unknown>:0"]; } } 12160->12161; 12161->12162; 12162->12163; 12163->12164; 12164->12165; 12165->12166; 12166->12167; 12167->12168; 12168->12169; 12169->12170; 12170->12171; 12171->12172; 12172->12173; 12173->12174; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time