3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1534 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 341
Associated Job Ids:
1534
DAG Visualization
digraph G { subgraph clusterstage_1534 { label="Stage 1534"; 16111 [labelType="html" label="SQLExecutionRDD [16111]<br>run at <unknown>:0"]; 16119 [labelType="html" label="SQLExecutionRDD [16119]<br>toJavaRDD at <unknown>:0"]; 16121 [labelType="html" label="PythonRDD [16121]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36054 { label="MicroBatchScan"; 16107 [labelType="html" label="DataSourceRDD [16107]<br>run at <unknown>:0"]; 16108 [labelType="html" label="MapPartitionsRDD [16108]<br>run at <unknown>:0"]; } subgraph cluster36053 { label="Project"; 16109 [labelType="html" label="MapPartitionsRDD [16109]<br>run at <unknown>:0"]; } subgraph cluster36049 { label="WholeStageCodegen (1)"; 16110 [labelType="html" label="MapPartitionsRDD [16110]<br>run at <unknown>:0"]; } subgraph cluster36063 { label="WholeStageCodegen (1)"; 16112 [labelType="html" label="MapPartitionsRDD [16112]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36062 { label="Filter"; 16113 [labelType="html" label="MapPartitionsRDD [16113]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36061 { label="Project"; 16114 [labelType="html" label="MapPartitionsRDD [16114]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36058 { label="WholeStageCodegen (2)"; 16115 [labelType="html" label="MapPartitionsRDD [16115]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36057 { label="Project"; 16116 [labelType="html" label="MapPartitionsRDD [16116]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36056 { label="DeserializeToObject"; 16117 [labelType="html" label="MapPartitionsRDD [16117]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36055 { label="MapPartitions"; 16118 [labelType="html" label="MapPartitionsRDD [16118]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36065 { label="mapPartitions"; 16120 [labelType="html" label="MapPartitionsRDD [16120]<br>toJavaRDD at <unknown>:0"]; } } 16107->16108; 16108->16109; 16109->16110; 16110->16111; 16111->16112; 16112->16113; 16113->16114; 16114->16115; 16115->16116; 16116->16117; 16117->16118; 16118->16119; 16119->16120; 16120->16121; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time