3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1724 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 519
Associated Job Ids:
1725
DAG Visualization
digraph G { subgraph clusterstage_1724 { label="Stage 1724"; 18113 [labelType="html" label="SQLExecutionRDD [18113]<br>run at <unknown>:0"]; 18121 [labelType="html" label="SQLExecutionRDD [18121]<br>toJavaRDD at <unknown>:0"]; 18123 [labelType="html" label="PythonRDD [18123]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster40523 { label="MicroBatchScan"; 18109 [labelType="html" label="DataSourceRDD [18109]<br>run at <unknown>:0"]; 18110 [labelType="html" label="MapPartitionsRDD [18110]<br>run at <unknown>:0"]; } subgraph cluster40522 { label="Project"; 18111 [labelType="html" label="MapPartitionsRDD [18111]<br>run at <unknown>:0"]; } subgraph cluster40518 { label="WholeStageCodegen (1)"; 18112 [labelType="html" label="MapPartitionsRDD [18112]<br>run at <unknown>:0"]; } subgraph cluster40532 { label="WholeStageCodegen (1)"; 18114 [labelType="html" label="MapPartitionsRDD [18114]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40531 { label="Filter"; 18115 [labelType="html" label="MapPartitionsRDD [18115]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40530 { label="Project"; 18116 [labelType="html" label="MapPartitionsRDD [18116]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40527 { label="WholeStageCodegen (2)"; 18117 [labelType="html" label="MapPartitionsRDD [18117]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40526 { label="Project"; 18118 [labelType="html" label="MapPartitionsRDD [18118]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40525 { label="DeserializeToObject"; 18119 [labelType="html" label="MapPartitionsRDD [18119]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40524 { label="MapPartitions"; 18120 [labelType="html" label="MapPartitionsRDD [18120]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster40534 { label="mapPartitions"; 18122 [labelType="html" label="MapPartitionsRDD [18122]<br>toJavaRDD at <unknown>:0"]; } } 18109->18110; 18110->18111; 18111->18112; 18112->18113; 18113->18114; 18114->18115; 18115->18116; 18116->18117; 18117->18118; 18118->18119; 18119->18120; 18120->18121; 18121->18122; 18122->18123; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time