3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1520 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 444
Associated Job Ids:
1520
DAG Visualization
digraph G { subgraph clusterstage_1520 { label="Stage 1520"; 15964 [labelType="html" label="SQLExecutionRDD [15964]<br>run at <unknown>:0"]; 15972 [labelType="html" label="SQLExecutionRDD [15972]<br>toJavaRDD at <unknown>:0"]; 15974 [labelType="html" label="PythonRDD [15974]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster35725 { label="MicroBatchScan"; 15960 [labelType="html" label="DataSourceRDD [15960]<br>run at <unknown>:0"]; 15961 [labelType="html" label="MapPartitionsRDD [15961]<br>run at <unknown>:0"]; } subgraph cluster35724 { label="Project"; 15962 [labelType="html" label="MapPartitionsRDD [15962]<br>run at <unknown>:0"]; } subgraph cluster35720 { label="WholeStageCodegen (1)"; 15963 [labelType="html" label="MapPartitionsRDD [15963]<br>run at <unknown>:0"]; } subgraph cluster35734 { label="WholeStageCodegen (1)"; 15965 [labelType="html" label="MapPartitionsRDD [15965]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35733 { label="Filter"; 15966 [labelType="html" label="MapPartitionsRDD [15966]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35732 { label="Project"; 15967 [labelType="html" label="MapPartitionsRDD [15967]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35729 { label="WholeStageCodegen (2)"; 15968 [labelType="html" label="MapPartitionsRDD [15968]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35728 { label="Project"; 15969 [labelType="html" label="MapPartitionsRDD [15969]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35727 { label="DeserializeToObject"; 15970 [labelType="html" label="MapPartitionsRDD [15970]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35726 { label="MapPartitions"; 15971 [labelType="html" label="MapPartitionsRDD [15971]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35736 { label="mapPartitions"; 15973 [labelType="html" label="MapPartitionsRDD [15973]<br>toJavaRDD at <unknown>:0"]; } } 15960->15961; 15961->15962; 15962->15963; 15963->15964; 15964->15965; 15965->15966; 15966->15967; 15967->15968; 15968->15969; 15969->15970; 15970->15971; 15971->15972; 15972->15973; 15973->15974; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time