3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1946 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 394
Associated Job Ids:
1947
DAG Visualization
digraph G { subgraph clusterstage_1946 { label="Stage 1946"; 20438 [labelType="html" label="SQLExecutionRDD [20438]<br>run at <unknown>:0"]; 20452 [labelType="html" label="SQLExecutionRDD [20452]<br>toJavaRDD at <unknown>:0"]; 20454 [labelType="html" label="PythonRDD [20454]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster45732 { label="MicroBatchScan"; 20434 [labelType="html" label="DataSourceRDD [20434]<br>run at <unknown>:0"]; 20435 [labelType="html" label="MapPartitionsRDD [20435]<br>run at <unknown>:0"]; } subgraph cluster45731 { label="Project"; 20436 [labelType="html" label="MapPartitionsRDD [20436]<br>run at <unknown>:0"]; } subgraph cluster45727 { label="WholeStageCodegen (1)"; 20437 [labelType="html" label="MapPartitionsRDD [20437]<br>run at <unknown>:0"]; } subgraph cluster45749 { label="WholeStageCodegen (1)"; 20445 [labelType="html" label="MapPartitionsRDD [20445]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45748 { label="Filter"; 20446 [labelType="html" label="MapPartitionsRDD [20446]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45747 { label="Project"; 20447 [labelType="html" label="MapPartitionsRDD [20447]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45744 { label="WholeStageCodegen (2)"; 20448 [labelType="html" label="MapPartitionsRDD [20448]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45743 { label="Project"; 20449 [labelType="html" label="MapPartitionsRDD [20449]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45742 { label="DeserializeToObject"; 20450 [labelType="html" label="MapPartitionsRDD [20450]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45741 { label="MapPartitions"; 20451 [labelType="html" label="MapPartitionsRDD [20451]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45751 { label="mapPartitions"; 20453 [labelType="html" label="MapPartitionsRDD [20453]<br>toJavaRDD at <unknown>:0"]; } } 20434->20435; 20435->20436; 20436->20437; 20437->20438; 20438->20445; 20445->20446; 20446->20447; 20447->20448; 20448->20449; 20449->20450; 20450->20451; 20451->20452; 20452->20453; 20453->20454; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time