3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1621 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 470
Associated Job Ids:
1622
DAG Visualization
digraph G { subgraph clusterstage_1621 { label="Stage 1621"; 17036 [labelType="html" label="SQLExecutionRDD [17036]<br>run at <unknown>:0"]; 17044 [labelType="html" label="SQLExecutionRDD [17044]<br>toJavaRDD at <unknown>:0"]; 17046 [labelType="html" label="PythonRDD [17046]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster38118 { label="MicroBatchScan"; 17032 [labelType="html" label="DataSourceRDD [17032]<br>run at <unknown>:0"]; 17033 [labelType="html" label="MapPartitionsRDD [17033]<br>run at <unknown>:0"]; } subgraph cluster38117 { label="Project"; 17034 [labelType="html" label="MapPartitionsRDD [17034]<br>run at <unknown>:0"]; } subgraph cluster38113 { label="WholeStageCodegen (1)"; 17035 [labelType="html" label="MapPartitionsRDD [17035]<br>run at <unknown>:0"]; } subgraph cluster38127 { label="WholeStageCodegen (1)"; 17037 [labelType="html" label="MapPartitionsRDD [17037]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38126 { label="Filter"; 17038 [labelType="html" label="MapPartitionsRDD [17038]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38125 { label="Project"; 17039 [labelType="html" label="MapPartitionsRDD [17039]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38122 { label="WholeStageCodegen (2)"; 17040 [labelType="html" label="MapPartitionsRDD [17040]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38121 { label="Project"; 17041 [labelType="html" label="MapPartitionsRDD [17041]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38120 { label="DeserializeToObject"; 17042 [labelType="html" label="MapPartitionsRDD [17042]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38119 { label="MapPartitions"; 17043 [labelType="html" label="MapPartitionsRDD [17043]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38129 { label="mapPartitions"; 17045 [labelType="html" label="MapPartitionsRDD [17045]<br>toJavaRDD at <unknown>:0"]; } } 17032->17033; 17033->17034; 17034->17035; 17035->17036; 17036->17037; 17037->17038; 17038->17039; 17039->17040; 17040->17041; 17041->17042; 17042->17043; 17043->17044; 17044->17045; 17045->17046; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time