3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1201 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 522
Associated Job Ids:
1202
DAG Visualization
digraph G { subgraph clusterstage_1201 { label="Stage 1201"; 12626 [labelType="html" label="SQLExecutionRDD [12626]<br>run at <unknown>:0"]; 12634 [labelType="html" label="SQLExecutionRDD [12634]<br>toJavaRDD at <unknown>:0"]; 12636 [labelType="html" label="PythonRDD [12636]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster28248 { label="MicroBatchScan"; 12622 [labelType="html" label="DataSourceRDD [12622]<br>run at <unknown>:0"]; 12623 [labelType="html" label="MapPartitionsRDD [12623]<br>run at <unknown>:0"]; } subgraph cluster28247 { label="Project"; 12624 [labelType="html" label="MapPartitionsRDD [12624]<br>run at <unknown>:0"]; } subgraph cluster28243 { label="WholeStageCodegen (1)"; 12625 [labelType="html" label="MapPartitionsRDD [12625]<br>run at <unknown>:0"]; } subgraph cluster28257 { label="WholeStageCodegen (1)"; 12627 [labelType="html" label="MapPartitionsRDD [12627]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28256 { label="Filter"; 12628 [labelType="html" label="MapPartitionsRDD [12628]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28255 { label="Project"; 12629 [labelType="html" label="MapPartitionsRDD [12629]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28252 { label="WholeStageCodegen (2)"; 12630 [labelType="html" label="MapPartitionsRDD [12630]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28251 { label="Project"; 12631 [labelType="html" label="MapPartitionsRDD [12631]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28250 { label="DeserializeToObject"; 12632 [labelType="html" label="MapPartitionsRDD [12632]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28249 { label="MapPartitions"; 12633 [labelType="html" label="MapPartitionsRDD [12633]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster28259 { label="mapPartitions"; 12635 [labelType="html" label="MapPartitionsRDD [12635]<br>toJavaRDD at <unknown>:0"]; } } 12622->12623; 12623->12624; 12624->12625; 12625->12626; 12626->12627; 12627->12628; 12628->12629; 12629->12630; 12630->12631; 12631->12632; 12632->12633; 12633->12634; 12634->12635; 12635->12636; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time