3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1540 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 480
Associated Job Ids:
1540
DAG Visualization
digraph G { subgraph clusterstage_1540 { label="Stage 1540"; 16174 [labelType="html" label="SQLExecutionRDD [16174]<br>run at <unknown>:0"]; 16182 [labelType="html" label="SQLExecutionRDD [16182]<br>toJavaRDD at <unknown>:0"]; 16184 [labelType="html" label="PythonRDD [16184]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36195 { label="MicroBatchScan"; 16170 [labelType="html" label="DataSourceRDD [16170]<br>run at <unknown>:0"]; 16171 [labelType="html" label="MapPartitionsRDD [16171]<br>run at <unknown>:0"]; } subgraph cluster36194 { label="Project"; 16172 [labelType="html" label="MapPartitionsRDD [16172]<br>run at <unknown>:0"]; } subgraph cluster36190 { label="WholeStageCodegen (1)"; 16173 [labelType="html" label="MapPartitionsRDD [16173]<br>run at <unknown>:0"]; } subgraph cluster36204 { label="WholeStageCodegen (1)"; 16175 [labelType="html" label="MapPartitionsRDD [16175]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36203 { label="Filter"; 16176 [labelType="html" label="MapPartitionsRDD [16176]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36202 { label="Project"; 16177 [labelType="html" label="MapPartitionsRDD [16177]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36199 { label="WholeStageCodegen (2)"; 16178 [labelType="html" label="MapPartitionsRDD [16178]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36198 { label="Project"; 16179 [labelType="html" label="MapPartitionsRDD [16179]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36197 { label="DeserializeToObject"; 16180 [labelType="html" label="MapPartitionsRDD [16180]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36196 { label="MapPartitions"; 16181 [labelType="html" label="MapPartitionsRDD [16181]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36206 { label="mapPartitions"; 16183 [labelType="html" label="MapPartitionsRDD [16183]<br>toJavaRDD at <unknown>:0"]; } } 16170->16171; 16171->16172; 16172->16173; 16173->16174; 16174->16175; 16175->16176; 16176->16177; 16177->16178; 16178->16179; 16179->16180; 16180->16181; 16181->16182; 16182->16183; 16183->16184; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time