3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1156 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 477
Associated Job Ids:
1157
DAG Visualization
digraph G { subgraph clusterstage_1156 { label="Stage 1156"; 12149 [labelType="html" label="SQLExecutionRDD [12149]<br>run at <unknown>:0"]; 12157 [labelType="html" label="SQLExecutionRDD [12157]<br>toJavaRDD at <unknown>:0"]; 12159 [labelType="html" label="PythonRDD [12159]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster27175 { label="MicroBatchScan"; 12145 [labelType="html" label="DataSourceRDD [12145]<br>run at <unknown>:0"]; 12146 [labelType="html" label="MapPartitionsRDD [12146]<br>run at <unknown>:0"]; } subgraph cluster27174 { label="Project"; 12147 [labelType="html" label="MapPartitionsRDD [12147]<br>run at <unknown>:0"]; } subgraph cluster27170 { label="WholeStageCodegen (1)"; 12148 [labelType="html" label="MapPartitionsRDD [12148]<br>run at <unknown>:0"]; } subgraph cluster27184 { label="WholeStageCodegen (1)"; 12150 [labelType="html" label="MapPartitionsRDD [12150]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27183 { label="Filter"; 12151 [labelType="html" label="MapPartitionsRDD [12151]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27182 { label="Project"; 12152 [labelType="html" label="MapPartitionsRDD [12152]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27179 { label="WholeStageCodegen (2)"; 12153 [labelType="html" label="MapPartitionsRDD [12153]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27178 { label="Project"; 12154 [labelType="html" label="MapPartitionsRDD [12154]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27177 { label="DeserializeToObject"; 12155 [labelType="html" label="MapPartitionsRDD [12155]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27176 { label="MapPartitions"; 12156 [labelType="html" label="MapPartitionsRDD [12156]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster27186 { label="mapPartitions"; 12158 [labelType="html" label="MapPartitionsRDD [12158]<br>toJavaRDD at <unknown>:0"]; } } 12145->12146; 12146->12147; 12147->12148; 12148->12149; 12149->12150; 12150->12151; 12151->12152; 12152->12153; 12153->12154; 12154->12155; 12155->12156; 12156->12157; 12157->12158; 12158->12159; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time