3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1506 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 443
Associated Job Ids:
1506
DAG Visualization
digraph G { subgraph clusterstage_1506 { label="Stage 1506"; 15817 [labelType="html" label="SQLExecutionRDD [15817]<br>run at <unknown>:0"]; 15825 [labelType="html" label="SQLExecutionRDD [15825]<br>toJavaRDD at <unknown>:0"]; 15827 [labelType="html" label="PythonRDD [15827]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster35396 { label="MicroBatchScan"; 15813 [labelType="html" label="DataSourceRDD [15813]<br>run at <unknown>:0"]; 15814 [labelType="html" label="MapPartitionsRDD [15814]<br>run at <unknown>:0"]; } subgraph cluster35395 { label="Project"; 15815 [labelType="html" label="MapPartitionsRDD [15815]<br>run at <unknown>:0"]; } subgraph cluster35391 { label="WholeStageCodegen (1)"; 15816 [labelType="html" label="MapPartitionsRDD [15816]<br>run at <unknown>:0"]; } subgraph cluster35405 { label="WholeStageCodegen (1)"; 15818 [labelType="html" label="MapPartitionsRDD [15818]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35404 { label="Filter"; 15819 [labelType="html" label="MapPartitionsRDD [15819]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35403 { label="Project"; 15820 [labelType="html" label="MapPartitionsRDD [15820]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35400 { label="WholeStageCodegen (2)"; 15821 [labelType="html" label="MapPartitionsRDD [15821]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35399 { label="Project"; 15822 [labelType="html" label="MapPartitionsRDD [15822]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35398 { label="DeserializeToObject"; 15823 [labelType="html" label="MapPartitionsRDD [15823]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35397 { label="MapPartitions"; 15824 [labelType="html" label="MapPartitionsRDD [15824]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35407 { label="mapPartitions"; 15826 [labelType="html" label="MapPartitionsRDD [15826]<br>toJavaRDD at <unknown>:0"]; } } 15813->15814; 15814->15815; 15815->15816; 15816->15817; 15817->15818; 15818->15819; 15819->15820; 15820->15821; 15821->15822; 15822->15823; 15823->15824; 15824->15825; 15825->15826; 15826->15827; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time