3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1793 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 450
Associated Job Ids:
1793
DAG Visualization
digraph G { subgraph clusterstage_1793 { label="Stage 1793"; 18820 [labelType="html" label="SQLExecutionRDD [18820]<br>run at <unknown>:0"]; 18834 [labelType="html" label="SQLExecutionRDD [18834]<br>toJavaRDD at <unknown>:0"]; 18836 [labelType="html" label="PythonRDD [18836]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster42117 { label="MicroBatchScan"; 18816 [labelType="html" label="DataSourceRDD [18816]<br>run at <unknown>:0"]; 18817 [labelType="html" label="MapPartitionsRDD [18817]<br>run at <unknown>:0"]; } subgraph cluster42116 { label="Project"; 18818 [labelType="html" label="MapPartitionsRDD [18818]<br>run at <unknown>:0"]; } subgraph cluster42112 { label="WholeStageCodegen (1)"; 18819 [labelType="html" label="MapPartitionsRDD [18819]<br>run at <unknown>:0"]; } subgraph cluster42134 { label="WholeStageCodegen (1)"; 18827 [labelType="html" label="MapPartitionsRDD [18827]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42133 { label="Filter"; 18828 [labelType="html" label="MapPartitionsRDD [18828]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42132 { label="Project"; 18829 [labelType="html" label="MapPartitionsRDD [18829]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42129 { label="WholeStageCodegen (2)"; 18830 [labelType="html" label="MapPartitionsRDD [18830]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42128 { label="Project"; 18831 [labelType="html" label="MapPartitionsRDD [18831]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42127 { label="DeserializeToObject"; 18832 [labelType="html" label="MapPartitionsRDD [18832]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42126 { label="MapPartitions"; 18833 [labelType="html" label="MapPartitionsRDD [18833]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42136 { label="mapPartitions"; 18835 [labelType="html" label="MapPartitionsRDD [18835]<br>toJavaRDD at <unknown>:0"]; } } 18816->18817; 18817->18818; 18818->18819; 18819->18820; 18820->18827; 18827->18828; 18828->18829; 18829->18830; 18830->18831; 18831->18832; 18832->18833; 18833->18834; 18834->18835; 18835->18836; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time