3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1533 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 492
Associated Job Ids:
1533
DAG Visualization
digraph G { subgraph clusterstage_1533 { label="Stage 1533"; 16096 [labelType="html" label="SQLExecutionRDD [16096]<br>run at <unknown>:0"]; 16104 [labelType="html" label="SQLExecutionRDD [16104]<br>toJavaRDD at <unknown>:0"]; 16106 [labelType="html" label="PythonRDD [16106]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36015 { label="MicroBatchScan"; 16092 [labelType="html" label="DataSourceRDD [16092]<br>run at <unknown>:0"]; 16093 [labelType="html" label="MapPartitionsRDD [16093]<br>run at <unknown>:0"]; } subgraph cluster36014 { label="Project"; 16094 [labelType="html" label="MapPartitionsRDD [16094]<br>run at <unknown>:0"]; } subgraph cluster36010 { label="WholeStageCodegen (1)"; 16095 [labelType="html" label="MapPartitionsRDD [16095]<br>run at <unknown>:0"]; } subgraph cluster36024 { label="WholeStageCodegen (1)"; 16097 [labelType="html" label="MapPartitionsRDD [16097]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36023 { label="Filter"; 16098 [labelType="html" label="MapPartitionsRDD [16098]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36022 { label="Project"; 16099 [labelType="html" label="MapPartitionsRDD [16099]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36019 { label="WholeStageCodegen (2)"; 16100 [labelType="html" label="MapPartitionsRDD [16100]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36018 { label="Project"; 16101 [labelType="html" label="MapPartitionsRDD [16101]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36017 { label="DeserializeToObject"; 16102 [labelType="html" label="MapPartitionsRDD [16102]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36016 { label="MapPartitions"; 16103 [labelType="html" label="MapPartitionsRDD [16103]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36026 { label="mapPartitions"; 16105 [labelType="html" label="MapPartitionsRDD [16105]<br>toJavaRDD at <unknown>:0"]; } } 16092->16093; 16093->16094; 16094->16095; 16095->16096; 16096->16097; 16097->16098; 16098->16099; 16099->16100; 16100->16101; 16101->16102; 16102->16103; 16103->16104; 16104->16105; 16105->16106; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time