3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1953 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 367
Associated Job Ids:
1954
DAG Visualization
digraph G { subgraph clusterstage_1953 { label="Stage 1953"; 20522 [labelType="html" label="SQLExecutionRDD [20522]<br>run at <unknown>:0"]; 20530 [labelType="html" label="SQLExecutionRDD [20530]<br>toJavaRDD at <unknown>:0"]; 20532 [labelType="html" label="PythonRDD [20532]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster45920 { label="MicroBatchScan"; 20518 [labelType="html" label="DataSourceRDD [20518]<br>run at <unknown>:0"]; 20519 [labelType="html" label="MapPartitionsRDD [20519]<br>run at <unknown>:0"]; } subgraph cluster45919 { label="Project"; 20520 [labelType="html" label="MapPartitionsRDD [20520]<br>run at <unknown>:0"]; } subgraph cluster45915 { label="WholeStageCodegen (1)"; 20521 [labelType="html" label="MapPartitionsRDD [20521]<br>run at <unknown>:0"]; } subgraph cluster45929 { label="WholeStageCodegen (1)"; 20523 [labelType="html" label="MapPartitionsRDD [20523]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45928 { label="Filter"; 20524 [labelType="html" label="MapPartitionsRDD [20524]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45927 { label="Project"; 20525 [labelType="html" label="MapPartitionsRDD [20525]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45924 { label="WholeStageCodegen (2)"; 20526 [labelType="html" label="MapPartitionsRDD [20526]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45923 { label="Project"; 20527 [labelType="html" label="MapPartitionsRDD [20527]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45922 { label="DeserializeToObject"; 20528 [labelType="html" label="MapPartitionsRDD [20528]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45921 { label="MapPartitions"; 20529 [labelType="html" label="MapPartitionsRDD [20529]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster45931 { label="mapPartitions"; 20531 [labelType="html" label="MapPartitionsRDD [20531]<br>toJavaRDD at <unknown>:0"]; } } 20518->20519; 20519->20520; 20520->20521; 20521->20522; 20522->20523; 20523->20524; 20524->20525; 20525->20526; 20526->20527; 20527->20528; 20528->20529; 20529->20530; 20530->20531; 20531->20532; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time