3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1617 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 473
Associated Job Ids:
1618
DAG Visualization
digraph G { subgraph clusterstage_1617 { label="Stage 1617"; 16994 [labelType="html" label="SQLExecutionRDD [16994]<br>run at <unknown>:0"]; 17002 [labelType="html" label="SQLExecutionRDD [17002]<br>toJavaRDD at <unknown>:0"]; 17004 [labelType="html" label="PythonRDD [17004]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster38024 { label="MicroBatchScan"; 16990 [labelType="html" label="DataSourceRDD [16990]<br>run at <unknown>:0"]; 16991 [labelType="html" label="MapPartitionsRDD [16991]<br>run at <unknown>:0"]; } subgraph cluster38023 { label="Project"; 16992 [labelType="html" label="MapPartitionsRDD [16992]<br>run at <unknown>:0"]; } subgraph cluster38019 { label="WholeStageCodegen (1)"; 16993 [labelType="html" label="MapPartitionsRDD [16993]<br>run at <unknown>:0"]; } subgraph cluster38033 { label="WholeStageCodegen (1)"; 16995 [labelType="html" label="MapPartitionsRDD [16995]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38032 { label="Filter"; 16996 [labelType="html" label="MapPartitionsRDD [16996]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38031 { label="Project"; 16997 [labelType="html" label="MapPartitionsRDD [16997]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38028 { label="WholeStageCodegen (2)"; 16998 [labelType="html" label="MapPartitionsRDD [16998]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38027 { label="Project"; 16999 [labelType="html" label="MapPartitionsRDD [16999]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38026 { label="DeserializeToObject"; 17000 [labelType="html" label="MapPartitionsRDD [17000]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38025 { label="MapPartitions"; 17001 [labelType="html" label="MapPartitionsRDD [17001]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster38035 { label="mapPartitions"; 17003 [labelType="html" label="MapPartitionsRDD [17003]<br>toJavaRDD at <unknown>:0"]; } } 16990->16991; 16991->16992; 16992->16993; 16993->16994; 16994->16995; 16995->16996; 16996->16997; 16997->16998; 16998->16999; 16999->17000; 17000->17001; 17001->17002; 17002->17003; 17003->17004; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time