3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1011 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 498
Associated Job Ids:
1012
DAG Visualization
digraph G { subgraph clusterstage_1011 { label="Stage 1011"; 10631 [labelType="html" label="SQLExecutionRDD [10631]<br>run at <unknown>:0"]; 10639 [labelType="html" label="SQLExecutionRDD [10639]<br>toJavaRDD at <unknown>:0"]; 10641 [labelType="html" label="PythonRDD [10641]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster23783 { label="MicroBatchScan"; 10627 [labelType="html" label="DataSourceRDD [10627]<br>run at <unknown>:0"]; 10628 [labelType="html" label="MapPartitionsRDD [10628]<br>run at <unknown>:0"]; } subgraph cluster23782 { label="Project"; 10629 [labelType="html" label="MapPartitionsRDD [10629]<br>run at <unknown>:0"]; } subgraph cluster23778 { label="WholeStageCodegen (1)"; 10630 [labelType="html" label="MapPartitionsRDD [10630]<br>run at <unknown>:0"]; } subgraph cluster23792 { label="WholeStageCodegen (1)"; 10632 [labelType="html" label="MapPartitionsRDD [10632]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23791 { label="Filter"; 10633 [labelType="html" label="MapPartitionsRDD [10633]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23790 { label="Project"; 10634 [labelType="html" label="MapPartitionsRDD [10634]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23787 { label="WholeStageCodegen (2)"; 10635 [labelType="html" label="MapPartitionsRDD [10635]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23786 { label="Project"; 10636 [labelType="html" label="MapPartitionsRDD [10636]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23785 { label="DeserializeToObject"; 10637 [labelType="html" label="MapPartitionsRDD [10637]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23784 { label="MapPartitions"; 10638 [labelType="html" label="MapPartitionsRDD [10638]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23794 { label="mapPartitions"; 10640 [labelType="html" label="MapPartitionsRDD [10640]<br>toJavaRDD at <unknown>:0"]; } } 10627->10628; 10628->10629; 10629->10630; 10630->10631; 10631->10632; 10632->10633; 10633->10634; 10634->10635; 10635->10636; 10636->10637; 10637->10638; 10638->10639; 10639->10640; 10640->10641; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time