3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1631 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 492
Associated Job Ids:
1632
DAG Visualization
digraph G { subgraph clusterstage_1631 { label="Stage 1631"; 17141 [labelType="html" label="SQLExecutionRDD [17141]<br>run at <unknown>:0"]; 17149 [labelType="html" label="SQLExecutionRDD [17149]<br>toJavaRDD at <unknown>:0"]; 17151 [labelType="html" label="PythonRDD [17151]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster38353 { label="MicroBatchScan"; 17137 [labelType="html" label="DataSourceRDD [17137]<br>run at <unknown>:0"]; 17138 [labelType="html" label="MapPartitionsRDD [17138]<br>run at <unknown>:0"]; } subgraph cluster38352 { label="Project"; 17139 [labelType="html" label="MapPartitionsRDD [17139]<br>run at <unknown>:0"]; } subgraph cluster38348 { label="WholeStageCodegen (1)"; 17140 [labelType="html" label="MapPartitionsRDD [17140]<br>run at <unknown>:0"]; } subgraph cluster38362 { label="WholeStageCodegen (1)"; 17142 [labelType="html" label="MapPartitionsRDD [17142]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38361 { label="Filter"; 17143 [labelType="html" label="MapPartitionsRDD [17143]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38360 { label="Project"; 17144 [labelType="html" label="MapPartitionsRDD [17144]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38357 { label="WholeStageCodegen (2)"; 17145 [labelType="html" label="MapPartitionsRDD [17145]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38356 { label="Project"; 17146 [labelType="html" label="MapPartitionsRDD [17146]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38355 { label="DeserializeToObject"; 17147 [labelType="html" label="MapPartitionsRDD [17147]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38354 { label="MapPartitions"; 17148 [labelType="html" label="MapPartitionsRDD [17148]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38364 { label="mapPartitions"; 17150 [labelType="html" label="MapPartitionsRDD [17150]<br>toJavaRDD at <unknown>:0"]; } } 17137->17138; 17138->17139; 17139->17140; 17140->17141; 17141->17142; 17142->17143; 17143->17144; 17144->17145; 17145->17146; 17146->17147; 17147->17148; 17148->17149; 17149->17150; 17150->17151; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time