3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1821 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 493
Associated Job Ids:
1821
DAG Visualization
digraph G { subgraph clusterstage_1821 { label="Stage 1821"; 19114 [labelType="html" label="SQLExecutionRDD [19114]<br>run at <unknown>:0"]; 19128 [labelType="html" label="SQLExecutionRDD [19128]<br>toJavaRDD at <unknown>:0"]; 19130 [labelType="html" label="PythonRDD [19130]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster42775 { label="MicroBatchScan"; 19110 [labelType="html" label="DataSourceRDD [19110]<br>run at <unknown>:0"]; 19111 [labelType="html" label="MapPartitionsRDD [19111]<br>run at <unknown>:0"]; } subgraph cluster42774 { label="Project"; 19112 [labelType="html" label="MapPartitionsRDD [19112]<br>run at <unknown>:0"]; } subgraph cluster42770 { label="WholeStageCodegen (1)"; 19113 [labelType="html" label="MapPartitionsRDD [19113]<br>run at <unknown>:0"]; } subgraph cluster42792 { label="WholeStageCodegen (1)"; 19121 [labelType="html" label="MapPartitionsRDD [19121]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42791 { label="Filter"; 19122 [labelType="html" label="MapPartitionsRDD [19122]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42790 { label="Project"; 19123 [labelType="html" label="MapPartitionsRDD [19123]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42787 { label="WholeStageCodegen (2)"; 19124 [labelType="html" label="MapPartitionsRDD [19124]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42786 { label="Project"; 19125 [labelType="html" label="MapPartitionsRDD [19125]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42785 { label="DeserializeToObject"; 19126 [labelType="html" label="MapPartitionsRDD [19126]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42784 { label="MapPartitions"; 19127 [labelType="html" label="MapPartitionsRDD [19127]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster42794 { label="mapPartitions"; 19129 [labelType="html" label="MapPartitionsRDD [19129]<br>toJavaRDD at <unknown>:0"]; } } 19110->19111; 19111->19112; 19112->19113; 19113->19114; 19114->19121; 19121->19122; 19122->19123; 19123->19124; 19124->19125; 19125->19126; 19126->19127; 19127->19128; 19128->19129; 19129->19130; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time