3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1007 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 497
Associated Job Ids:
1008
DAG Visualization
digraph G { subgraph clusterstage_1007 { label="Stage 1007"; 10589 [labelType="html" label="SQLExecutionRDD [10589]<br>run at <unknown>:0"]; 10597 [labelType="html" label="SQLExecutionRDD [10597]<br>toJavaRDD at <unknown>:0"]; 10599 [labelType="html" label="PythonRDD [10599]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster23689 { label="MicroBatchScan"; 10585 [labelType="html" label="DataSourceRDD [10585]<br>run at <unknown>:0"]; 10586 [labelType="html" label="MapPartitionsRDD [10586]<br>run at <unknown>:0"]; } subgraph cluster23688 { label="Project"; 10587 [labelType="html" label="MapPartitionsRDD [10587]<br>run at <unknown>:0"]; } subgraph cluster23684 { label="WholeStageCodegen (1)"; 10588 [labelType="html" label="MapPartitionsRDD [10588]<br>run at <unknown>:0"]; } subgraph cluster23698 { label="WholeStageCodegen (1)"; 10590 [labelType="html" label="MapPartitionsRDD [10590]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23697 { label="Filter"; 10591 [labelType="html" label="MapPartitionsRDD [10591]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23696 { label="Project"; 10592 [labelType="html" label="MapPartitionsRDD [10592]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23693 { label="WholeStageCodegen (2)"; 10593 [labelType="html" label="MapPartitionsRDD [10593]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23692 { label="Project"; 10594 [labelType="html" label="MapPartitionsRDD [10594]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23691 { label="DeserializeToObject"; 10595 [labelType="html" label="MapPartitionsRDD [10595]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23690 { label="MapPartitions"; 10596 [labelType="html" label="MapPartitionsRDD [10596]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster23700 { label="mapPartitions"; 10598 [labelType="html" label="MapPartitionsRDD [10598]<br>toJavaRDD at <unknown>:0"]; } } 10585->10586; 10586->10587; 10587->10588; 10588->10589; 10589->10590; 10590->10591; 10591->10592; 10592->10593; 10593->10594; 10594->10595; 10595->10596; 10596->10597; 10597->10598; 10598->10599; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time