3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1779 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 430
Associated Job Ids:
1779
DAG Visualization
digraph G { subgraph clusterstage_1779 { label="Stage 1779"; 18673 [labelType="html" label="SQLExecutionRDD [18673]<br>run at <unknown>:0"]; 18687 [labelType="html" label="SQLExecutionRDD [18687]<br>toJavaRDD at <unknown>:0"]; 18689 [labelType="html" label="PythonRDD [18689]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster41788 { label="MicroBatchScan"; 18669 [labelType="html" label="DataSourceRDD [18669]<br>run at <unknown>:0"]; 18670 [labelType="html" label="MapPartitionsRDD [18670]<br>run at <unknown>:0"]; } subgraph cluster41787 { label="Project"; 18671 [labelType="html" label="MapPartitionsRDD [18671]<br>run at <unknown>:0"]; } subgraph cluster41783 { label="WholeStageCodegen (1)"; 18672 [labelType="html" label="MapPartitionsRDD [18672]<br>run at <unknown>:0"]; } subgraph cluster41805 { label="WholeStageCodegen (1)"; 18680 [labelType="html" label="MapPartitionsRDD [18680]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41804 { label="Filter"; 18681 [labelType="html" label="MapPartitionsRDD [18681]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41803 { label="Project"; 18682 [labelType="html" label="MapPartitionsRDD [18682]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41800 { label="WholeStageCodegen (2)"; 18683 [labelType="html" label="MapPartitionsRDD [18683]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41799 { label="Project"; 18684 [labelType="html" label="MapPartitionsRDD [18684]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41798 { label="DeserializeToObject"; 18685 [labelType="html" label="MapPartitionsRDD [18685]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41797 { label="MapPartitions"; 18686 [labelType="html" label="MapPartitionsRDD [18686]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41807 { label="mapPartitions"; 18688 [labelType="html" label="MapPartitionsRDD [18688]<br>toJavaRDD at <unknown>:0"]; } } 18669->18670; 18670->18671; 18671->18672; 18672->18673; 18673->18680; 18680->18681; 18681->18682; 18682->18683; 18683->18684; 18684->18685; 18685->18686; 18686->18687; 18687->18688; 18688->18689; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time