3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 13 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 417
Associated Job Ids:
13
DAG Visualization
digraph G { subgraph clusterstage_13 { label="Stage 13"; 130 [labelType="html" label="SQLExecutionRDD [130]<br>run at <unknown>:0"]; 144 [labelType="html" label="SQLExecutionRDD [144]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; 146 [labelType="html" label="PythonRDD [146]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster293 { label="MicroBatchScan"; 126 [labelType="html" label="DataSourceRDD [126]<br>run at <unknown>:0"]; 127 [labelType="html" label="MapPartitionsRDD [127]<br>run at <unknown>:0"]; } subgraph cluster292 { label="Project"; 128 [labelType="html" label="MapPartitionsRDD [128]<br>run at <unknown>:0"]; } subgraph cluster282 { label="WholeStageCodegen (1)"; 129 [labelType="html" label="MapPartitionsRDD [129]<br>run at <unknown>:0"]; } subgraph cluster304 { label="WholeStageCodegen (1)"; 137 [labelType="html" label="MapPartitionsRDD [137]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster303 { label="Filter"; 138 [labelType="html" label="MapPartitionsRDD [138]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster302 { label="Project"; 139 [labelType="html" label="MapPartitionsRDD [139]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster299 { label="WholeStageCodegen (2)"; 140 [labelType="html" label="MapPartitionsRDD [140]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster298 { label="Project"; 141 [labelType="html" label="MapPartitionsRDD [141]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster297 { label="DeserializeToObject"; 142 [labelType="html" label="MapPartitionsRDD [142]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster296 { label="MapPartitions"; 143 [labelType="html" label="MapPartitionsRDD [143]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } subgraph cluster306 { label="mapPartitions"; 145 [labelType="html" label="MapPartitionsRDD [145]<br>toJavaRDD at NativeMethodAccessorImpl.java:0"]; } } 126->127; 127->128; 128->129; 129->130; 130->137; 137->138; 138->139; 139->140; 140->141; 141->142; 142->143; 143->144; 144->145; 145->146; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time