3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 970 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 477
Associated Job Ids:
970
DAG Visualization
digraph G { subgraph clusterstage_970 { label="Stage 970"; 10189 [labelType="html" label="SQLExecutionRDD [10189]<br>run at <unknown>:0"]; 10197 [labelType="html" label="SQLExecutionRDD [10197]<br>toJavaRDD at <unknown>:0"]; 10199 [labelType="html" label="PythonRDD [10199]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster22800 { label="MicroBatchScan"; 10185 [labelType="html" label="DataSourceRDD [10185]<br>run at <unknown>:0"]; 10186 [labelType="html" label="MapPartitionsRDD [10186]<br>run at <unknown>:0"]; } subgraph cluster22799 { label="Project"; 10187 [labelType="html" label="MapPartitionsRDD [10187]<br>run at <unknown>:0"]; } subgraph cluster22795 { label="WholeStageCodegen (1)"; 10188 [labelType="html" label="MapPartitionsRDD [10188]<br>run at <unknown>:0"]; } subgraph cluster22809 { label="WholeStageCodegen (1)"; 10190 [labelType="html" label="MapPartitionsRDD [10190]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22808 { label="Filter"; 10191 [labelType="html" label="MapPartitionsRDD [10191]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22807 { label="Project"; 10192 [labelType="html" label="MapPartitionsRDD [10192]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22804 { label="WholeStageCodegen (2)"; 10193 [labelType="html" label="MapPartitionsRDD [10193]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22803 { label="Project"; 10194 [labelType="html" label="MapPartitionsRDD [10194]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22802 { label="DeserializeToObject"; 10195 [labelType="html" label="MapPartitionsRDD [10195]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22801 { label="MapPartitions"; 10196 [labelType="html" label="MapPartitionsRDD [10196]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster22811 { label="mapPartitions"; 10198 [labelType="html" label="MapPartitionsRDD [10198]<br>toJavaRDD at <unknown>:0"]; } } 10185->10186; 10186->10187; 10187->10188; 10188->10189; 10189->10190; 10190->10191; 10191->10192; 10192->10193; 10193->10194; 10194->10195; 10195->10196; 10196->10197; 10197->10198; 10198->10199; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time