3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1561 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 482
Associated Job Ids:
1562
DAG Visualization
digraph G { subgraph clusterstage_1561 { label="Stage 1561"; 16406 [labelType="html" label="SQLExecutionRDD [16406]<br>run at <unknown>:0"]; 16414 [labelType="html" label="SQLExecutionRDD [16414]<br>toJavaRDD at <unknown>:0"]; 16416 [labelType="html" label="PythonRDD [16416]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster36708 { label="MicroBatchScan"; 16402 [labelType="html" label="DataSourceRDD [16402]<br>run at <unknown>:0"]; 16403 [labelType="html" label="MapPartitionsRDD [16403]<br>run at <unknown>:0"]; } subgraph cluster36707 { label="Project"; 16404 [labelType="html" label="MapPartitionsRDD [16404]<br>run at <unknown>:0"]; } subgraph cluster36703 { label="WholeStageCodegen (1)"; 16405 [labelType="html" label="MapPartitionsRDD [16405]<br>run at <unknown>:0"]; } subgraph cluster36717 { label="WholeStageCodegen (1)"; 16407 [labelType="html" label="MapPartitionsRDD [16407]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36716 { label="Filter"; 16408 [labelType="html" label="MapPartitionsRDD [16408]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36715 { label="Project"; 16409 [labelType="html" label="MapPartitionsRDD [16409]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36712 { label="WholeStageCodegen (2)"; 16410 [labelType="html" label="MapPartitionsRDD [16410]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36711 { label="Project"; 16411 [labelType="html" label="MapPartitionsRDD [16411]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36710 { label="DeserializeToObject"; 16412 [labelType="html" label="MapPartitionsRDD [16412]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36709 { label="MapPartitions"; 16413 [labelType="html" label="MapPartitionsRDD [16413]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster36719 { label="mapPartitions"; 16415 [labelType="html" label="MapPartitionsRDD [16415]<br>toJavaRDD at <unknown>:0"]; } } 16402->16403; 16403->16404; 16404->16405; 16405->16406; 16406->16407; 16407->16408; 16408->16409; 16409->16410; 16410->16411; 16411->16412; 16412->16413; 16413->16414; 16414->16415; 16415->16416; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time