3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1565 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 407
Associated Job Ids:
1566
DAG Visualization
digraph G { subgraph clusterstage_1565 { label="Stage 1565"; 16448 [labelType="html" label="SQLExecutionRDD [16448]<br>run at <unknown>:0"]; 16456 [labelType="html" label="SQLExecutionRDD [16456]<br>toJavaRDD at <unknown>:0"]; 16458 [labelType="html" label="PythonRDD [16458]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36802 { label="MicroBatchScan"; 16444 [labelType="html" label="DataSourceRDD [16444]<br>run at <unknown>:0"]; 16445 [labelType="html" label="MapPartitionsRDD [16445]<br>run at <unknown>:0"]; } subgraph cluster36801 { label="Project"; 16446 [labelType="html" label="MapPartitionsRDD [16446]<br>run at <unknown>:0"]; } subgraph cluster36797 { label="WholeStageCodegen (1)"; 16447 [labelType="html" label="MapPartitionsRDD [16447]<br>run at <unknown>:0"]; } subgraph cluster36811 { label="WholeStageCodegen (1)"; 16449 [labelType="html" label="MapPartitionsRDD [16449]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36810 { label="Filter"; 16450 [labelType="html" label="MapPartitionsRDD [16450]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36809 { label="Project"; 16451 [labelType="html" label="MapPartitionsRDD [16451]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36806 { label="WholeStageCodegen (2)"; 16452 [labelType="html" label="MapPartitionsRDD [16452]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36805 { label="Project"; 16453 [labelType="html" label="MapPartitionsRDD [16453]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36804 { label="DeserializeToObject"; 16454 [labelType="html" label="MapPartitionsRDD [16454]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36803 { label="MapPartitions"; 16455 [labelType="html" label="MapPartitionsRDD [16455]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36813 { label="mapPartitions"; 16457 [labelType="html" label="MapPartitionsRDD [16457]<br>toJavaRDD at <unknown>:0"]; } } 16444->16445; 16445->16446; 16446->16447; 16447->16448; 16448->16449; 16449->16450; 16450->16451; 16451->16452; 16452->16453; 16453->16454; 16454->16455; 16455->16456; 16456->16457; 16457->16458; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time