3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 981 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 388
Associated Job Ids:
982
DAG Visualization
digraph G { subgraph clusterstage_981 { label="Stage 981"; 10316 [labelType="html" label="SQLExecutionRDD [10316]<br>run at <unknown>:0"]; 10324 [labelType="html" label="SQLExecutionRDD [10324]<br>toJavaRDD at <unknown>:0"]; 10326 [labelType="html" label="PythonRDD [10326]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster23078 { label="MicroBatchScan"; 10312 [labelType="html" label="DataSourceRDD [10312]<br>run at <unknown>:0"]; 10313 [labelType="html" label="MapPartitionsRDD [10313]<br>run at <unknown>:0"]; } subgraph cluster23077 { label="Project"; 10314 [labelType="html" label="MapPartitionsRDD [10314]<br>run at <unknown>:0"]; } subgraph cluster23073 { label="WholeStageCodegen (1)"; 10315 [labelType="html" label="MapPartitionsRDD [10315]<br>run at <unknown>:0"]; } subgraph cluster23087 { label="WholeStageCodegen (1)"; 10317 [labelType="html" label="MapPartitionsRDD [10317]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23086 { label="Filter"; 10318 [labelType="html" label="MapPartitionsRDD [10318]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23085 { label="Project"; 10319 [labelType="html" label="MapPartitionsRDD [10319]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23082 { label="WholeStageCodegen (2)"; 10320 [labelType="html" label="MapPartitionsRDD [10320]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23081 { label="Project"; 10321 [labelType="html" label="MapPartitionsRDD [10321]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23080 { label="DeserializeToObject"; 10322 [labelType="html" label="MapPartitionsRDD [10322]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23079 { label="MapPartitions"; 10323 [labelType="html" label="MapPartitionsRDD [10323]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23089 { label="mapPartitions"; 10325 [labelType="html" label="MapPartitionsRDD [10325]<br>toJavaRDD at <unknown>:0"]; } } 10312->10313; 10313->10314; 10314->10315; 10315->10316; 10316->10317; 10317->10318; 10318->10319; 10319->10320; 10320->10321; 10321->10322; 10322->10323; 10323->10324; 10324->10325; 10325->10326; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time