3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1458 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 457
Associated Job Ids:
1458
DAG Visualization
digraph G { subgraph clusterstage_1458 { label="Stage 1458"; 15313 [labelType="html" label="SQLExecutionRDD [15313]<br>run at <unknown>:0"]; 15321 [labelType="html" label="SQLExecutionRDD [15321]<br>toJavaRDD at <unknown>:0"]; 15326 [labelType="html" label="PythonRDD [15326]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster34268 { label="MicroBatchScan"; 15309 [labelType="html" label="DataSourceRDD [15309]<br>run at <unknown>:0"]; 15310 [labelType="html" label="MapPartitionsRDD [15310]<br>run at <unknown>:0"]; } subgraph cluster34267 { label="Project"; 15311 [labelType="html" label="MapPartitionsRDD [15311]<br>run at <unknown>:0"]; } subgraph cluster34263 { label="WholeStageCodegen (1)"; 15312 [labelType="html" label="MapPartitionsRDD [15312]<br>run at <unknown>:0"]; } subgraph cluster34277 { label="WholeStageCodegen (1)"; 15314 [labelType="html" label="MapPartitionsRDD [15314]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34276 { label="Filter"; 15315 [labelType="html" label="MapPartitionsRDD [15315]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34275 { label="Project"; 15316 [labelType="html" label="MapPartitionsRDD [15316]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34272 { label="WholeStageCodegen (2)"; 15317 [labelType="html" label="MapPartitionsRDD [15317]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34271 { label="Project"; 15318 [labelType="html" label="MapPartitionsRDD [15318]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34270 { label="DeserializeToObject"; 15319 [labelType="html" label="MapPartitionsRDD [15319]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34269 { label="MapPartitions"; 15320 [labelType="html" label="MapPartitionsRDD [15320]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34279 { label="mapPartitions"; 15322 [labelType="html" label="MapPartitionsRDD [15322]<br>toJavaRDD at <unknown>:0"]; } } 15309->15310; 15310->15311; 15311->15312; 15312->15313; 15313->15314; 15314->15315; 15315->15316; 15316->15317; 15317->15318; 15318->15319; 15319->15320; 15320->15321; 15321->15322; 15322->15326; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time