3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 580 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 588
Associated Job Ids:
580
DAG Visualization
digraph G { subgraph clusterstage_580 { label="Stage 580"; 6094 [labelType="html" label="SQLExecutionRDD [6094]<br>run at <unknown>:0"]; 6102 [labelType="html" label="SQLExecutionRDD [6102]<br>toJavaRDD at <unknown>:0"]; 6104 [labelType="html" label="PythonRDD [6104]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster13635 { label="MicroBatchScan"; 6090 [labelType="html" label="DataSourceRDD [6090]<br>run at <unknown>:0"]; 6091 [labelType="html" label="MapPartitionsRDD [6091]<br>run at <unknown>:0"]; } subgraph cluster13634 { label="Project"; 6092 [labelType="html" label="MapPartitionsRDD [6092]<br>run at <unknown>:0"]; } subgraph cluster13630 { label="WholeStageCodegen (1)"; 6093 [labelType="html" label="MapPartitionsRDD [6093]<br>run at <unknown>:0"]; } subgraph cluster13644 { label="WholeStageCodegen (1)"; 6095 [labelType="html" label="MapPartitionsRDD [6095]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13643 { label="Filter"; 6096 [labelType="html" label="MapPartitionsRDD [6096]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13642 { label="Project"; 6097 [labelType="html" label="MapPartitionsRDD [6097]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13639 { label="WholeStageCodegen (2)"; 6098 [labelType="html" label="MapPartitionsRDD [6098]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13638 { label="Project"; 6099 [labelType="html" label="MapPartitionsRDD [6099]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13637 { label="DeserializeToObject"; 6100 [labelType="html" label="MapPartitionsRDD [6100]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13636 { label="MapPartitions"; 6101 [labelType="html" label="MapPartitionsRDD [6101]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster13646 { label="mapPartitions"; 6103 [labelType="html" label="MapPartitionsRDD [6103]<br>toJavaRDD at <unknown>:0"]; } } 6090->6091; 6091->6092; 6092->6093; 6093->6094; 6094->6095; 6095->6096; 6096->6097; 6097->6098; 6098->6099; 6099->6100; 6100->6101; 6101->6102; 6102->6103; 6103->6104; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time