3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1573 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 406
Associated Job Ids:
1574
DAG Visualization
digraph G { subgraph clusterstage_1573 { label="Stage 1573"; 16532 [labelType="html" label="SQLExecutionRDD [16532]<br>run at <unknown>:0"]; 16540 [labelType="html" label="SQLExecutionRDD [16540]<br>toJavaRDD at <unknown>:0"]; 16542 [labelType="html" label="PythonRDD [16542]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36990 { label="MicroBatchScan"; 16528 [labelType="html" label="DataSourceRDD [16528]<br>run at <unknown>:0"]; 16529 [labelType="html" label="MapPartitionsRDD [16529]<br>run at <unknown>:0"]; } subgraph cluster36989 { label="Project"; 16530 [labelType="html" label="MapPartitionsRDD [16530]<br>run at <unknown>:0"]; } subgraph cluster36985 { label="WholeStageCodegen (1)"; 16531 [labelType="html" label="MapPartitionsRDD [16531]<br>run at <unknown>:0"]; } subgraph cluster36999 { label="WholeStageCodegen (1)"; 16533 [labelType="html" label="MapPartitionsRDD [16533]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36998 { label="Filter"; 16534 [labelType="html" label="MapPartitionsRDD [16534]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36997 { label="Project"; 16535 [labelType="html" label="MapPartitionsRDD [16535]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36994 { label="WholeStageCodegen (2)"; 16536 [labelType="html" label="MapPartitionsRDD [16536]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36993 { label="Project"; 16537 [labelType="html" label="MapPartitionsRDD [16537]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36992 { label="DeserializeToObject"; 16538 [labelType="html" label="MapPartitionsRDD [16538]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36991 { label="MapPartitions"; 16539 [labelType="html" label="MapPartitionsRDD [16539]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster37001 { label="mapPartitions"; 16541 [labelType="html" label="MapPartitionsRDD [16541]<br>toJavaRDD at <unknown>:0"]; } } 16528->16529; 16529->16530; 16530->16531; 16531->16532; 16532->16533; 16533->16534; 16534->16535; 16535->16536; 16536->16537; 16537->16538; 16538->16539; 16539->16540; 16540->16541; 16541->16542; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time