3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1472 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 389
Associated Job Ids:
1472
DAG Visualization
digraph G { subgraph clusterstage_1472 { label="Stage 1472"; 15460 [labelType="html" label="SQLExecutionRDD [15460]<br>run at <unknown>:0"]; 15468 [labelType="html" label="SQLExecutionRDD [15468]<br>toJavaRDD at <unknown>:0"]; 15470 [labelType="html" label="PythonRDD [15470]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster34597 { label="MicroBatchScan"; 15456 [labelType="html" label="DataSourceRDD [15456]<br>run at <unknown>:0"]; 15457 [labelType="html" label="MapPartitionsRDD [15457]<br>run at <unknown>:0"]; } subgraph cluster34596 { label="Project"; 15458 [labelType="html" label="MapPartitionsRDD [15458]<br>run at <unknown>:0"]; } subgraph cluster34592 { label="WholeStageCodegen (1)"; 15459 [labelType="html" label="MapPartitionsRDD [15459]<br>run at <unknown>:0"]; } subgraph cluster34606 { label="WholeStageCodegen (1)"; 15461 [labelType="html" label="MapPartitionsRDD [15461]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34605 { label="Filter"; 15462 [labelType="html" label="MapPartitionsRDD [15462]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34604 { label="Project"; 15463 [labelType="html" label="MapPartitionsRDD [15463]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34601 { label="WholeStageCodegen (2)"; 15464 [labelType="html" label="MapPartitionsRDD [15464]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34600 { label="Project"; 15465 [labelType="html" label="MapPartitionsRDD [15465]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34599 { label="DeserializeToObject"; 15466 [labelType="html" label="MapPartitionsRDD [15466]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34598 { label="MapPartitions"; 15467 [labelType="html" label="MapPartitionsRDD [15467]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34608 { label="mapPartitions"; 15469 [labelType="html" label="MapPartitionsRDD [15469]<br>toJavaRDD at <unknown>:0"]; } } 15456->15457; 15457->15458; 15458->15459; 15459->15460; 15460->15461; 15461->15462; 15462->15463; 15463->15464; 15464->15465; 15465->15466; 15466->15467; 15467->15468; 15468->15469; 15469->15470; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time