3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1030 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 471
Associated Job Ids:
1030
DAG Visualization
digraph G { subgraph clusterstage_1030 { label="Stage 1030"; 10819 [labelType="html" label="SQLExecutionRDD [10819]<br>run at <unknown>:0"]; 10827 [labelType="html" label="SQLExecutionRDD [10827]<br>toJavaRDD at <unknown>:0"]; 10829 [labelType="html" label="PythonRDD [10829]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster24210 { label="MicroBatchScan"; 10815 [labelType="html" label="DataSourceRDD [10815]<br>run at <unknown>:0"]; 10816 [labelType="html" label="MapPartitionsRDD [10816]<br>run at <unknown>:0"]; } subgraph cluster24209 { label="Project"; 10817 [labelType="html" label="MapPartitionsRDD [10817]<br>run at <unknown>:0"]; } subgraph cluster24205 { label="WholeStageCodegen (1)"; 10818 [labelType="html" label="MapPartitionsRDD [10818]<br>run at <unknown>:0"]; } subgraph cluster24219 { label="WholeStageCodegen (1)"; 10820 [labelType="html" label="MapPartitionsRDD [10820]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24218 { label="Filter"; 10821 [labelType="html" label="MapPartitionsRDD [10821]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24217 { label="Project"; 10822 [labelType="html" label="MapPartitionsRDD [10822]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24214 { label="WholeStageCodegen (2)"; 10823 [labelType="html" label="MapPartitionsRDD [10823]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24213 { label="Project"; 10824 [labelType="html" label="MapPartitionsRDD [10824]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24212 { label="DeserializeToObject"; 10825 [labelType="html" label="MapPartitionsRDD [10825]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24211 { label="MapPartitions"; 10826 [labelType="html" label="MapPartitionsRDD [10826]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24221 { label="mapPartitions"; 10828 [labelType="html" label="MapPartitionsRDD [10828]<br>toJavaRDD at <unknown>:0"]; } } 10815->10816; 10816->10817; 10817->10818; 10818->10819; 10819->10820; 10820->10821; 10821->10822; 10822->10823; 10823->10824; 10824->10825; 10825->10826; 10826->10827; 10827->10828; 10828->10829; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time