3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1080 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 486
Associated Job Ids:
1080
DAG Visualization
digraph G { subgraph clusterstage_1080 { label="Stage 1080"; 11344 [labelType="html" label="SQLExecutionRDD [11344]<br>run at <unknown>:0"]; 11352 [labelType="html" label="SQLExecutionRDD [11352]<br>toJavaRDD at <unknown>:0"]; 11354 [labelType="html" label="PythonRDD [11354]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster25385 { label="MicroBatchScan"; 11340 [labelType="html" label="DataSourceRDD [11340]<br>run at <unknown>:0"]; 11341 [labelType="html" label="MapPartitionsRDD [11341]<br>run at <unknown>:0"]; } subgraph cluster25384 { label="Project"; 11342 [labelType="html" label="MapPartitionsRDD [11342]<br>run at <unknown>:0"]; } subgraph cluster25380 { label="WholeStageCodegen (1)"; 11343 [labelType="html" label="MapPartitionsRDD [11343]<br>run at <unknown>:0"]; } subgraph cluster25394 { label="WholeStageCodegen (1)"; 11345 [labelType="html" label="MapPartitionsRDD [11345]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25393 { label="Filter"; 11346 [labelType="html" label="MapPartitionsRDD [11346]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25392 { label="Project"; 11347 [labelType="html" label="MapPartitionsRDD [11347]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25389 { label="WholeStageCodegen (2)"; 11348 [labelType="html" label="MapPartitionsRDD [11348]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25388 { label="Project"; 11349 [labelType="html" label="MapPartitionsRDD [11349]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25387 { label="DeserializeToObject"; 11350 [labelType="html" label="MapPartitionsRDD [11350]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25386 { label="MapPartitions"; 11351 [labelType="html" label="MapPartitionsRDD [11351]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster25396 { label="mapPartitions"; 11353 [labelType="html" label="MapPartitionsRDD [11353]<br>toJavaRDD at <unknown>:0"]; } } 11340->11341; 11341->11342; 11342->11343; 11343->11344; 11344->11345; 11345->11346; 11346->11347; 11347->11348; 11348->11349; 11349->11350; 11350->11351; 11351->11352; 11352->11353; 11353->11354; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time