3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1427 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 470
Associated Job Ids:
1428
DAG Visualization
digraph G { subgraph clusterstage_1427 { label="Stage 1427"; 14999 [labelType="html" label="SQLExecutionRDD [14999]<br>run at <unknown>:0"]; 15007 [labelType="html" label="SQLExecutionRDD [15007]<br>toJavaRDD at <unknown>:0"]; 15009 [labelType="html" label="PythonRDD [15009]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster33559 { label="MicroBatchScan"; 14995 [labelType="html" label="DataSourceRDD [14995]<br>run at <unknown>:0"]; 14996 [labelType="html" label="MapPartitionsRDD [14996]<br>run at <unknown>:0"]; } subgraph cluster33558 { label="Project"; 14997 [labelType="html" label="MapPartitionsRDD [14997]<br>run at <unknown>:0"]; } subgraph cluster33554 { label="WholeStageCodegen (1)"; 14998 [labelType="html" label="MapPartitionsRDD [14998]<br>run at <unknown>:0"]; } subgraph cluster33568 { label="WholeStageCodegen (1)"; 15000 [labelType="html" label="MapPartitionsRDD [15000]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33567 { label="Filter"; 15001 [labelType="html" label="MapPartitionsRDD [15001]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33566 { label="Project"; 15002 [labelType="html" label="MapPartitionsRDD [15002]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33563 { label="WholeStageCodegen (2)"; 15003 [labelType="html" label="MapPartitionsRDD [15003]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33562 { label="Project"; 15004 [labelType="html" label="MapPartitionsRDD [15004]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33561 { label="DeserializeToObject"; 15005 [labelType="html" label="MapPartitionsRDD [15005]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33560 { label="MapPartitions"; 15006 [labelType="html" label="MapPartitionsRDD [15006]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster33570 { label="mapPartitions"; 15008 [labelType="html" label="MapPartitionsRDD [15008]<br>toJavaRDD at <unknown>:0"]; } } 14995->14996; 14996->14997; 14997->14998; 14998->14999; 14999->15000; 15000->15001; 15001->15002; 15002->15003; 15003->15004; 15004->15005; 15005->15006; 15006->15007; 15007->15008; 15008->15009; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time