3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 488 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 426
Associated Job Ids:
488
DAG Visualization
digraph G { subgraph clusterstage_488 { label="Stage 488"; 5128 [labelType="html" label="SQLExecutionRDD [5128]<br>run at <unknown>:0"]; 5136 [labelType="html" label="SQLExecutionRDD [5136]<br>toJavaRDD at <unknown>:0"]; 5138 [labelType="html" label="PythonRDD [5138]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster11473 { label="MicroBatchScan"; 5124 [labelType="html" label="DataSourceRDD [5124]<br>run at <unknown>:0"]; 5125 [labelType="html" label="MapPartitionsRDD [5125]<br>run at <unknown>:0"]; } subgraph cluster11472 { label="Project"; 5126 [labelType="html" label="MapPartitionsRDD [5126]<br>run at <unknown>:0"]; } subgraph cluster11468 { label="WholeStageCodegen (1)"; 5127 [labelType="html" label="MapPartitionsRDD [5127]<br>run at <unknown>:0"]; } subgraph cluster11482 { label="WholeStageCodegen (1)"; 5129 [labelType="html" label="MapPartitionsRDD [5129]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11481 { label="Filter"; 5130 [labelType="html" label="MapPartitionsRDD [5130]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11480 { label="Project"; 5131 [labelType="html" label="MapPartitionsRDD [5131]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11477 { label="WholeStageCodegen (2)"; 5132 [labelType="html" label="MapPartitionsRDD [5132]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11476 { label="Project"; 5133 [labelType="html" label="MapPartitionsRDD [5133]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11475 { label="DeserializeToObject"; 5134 [labelType="html" label="MapPartitionsRDD [5134]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11474 { label="MapPartitions"; 5135 [labelType="html" label="MapPartitionsRDD [5135]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster11484 { label="mapPartitions"; 5137 [labelType="html" label="MapPartitionsRDD [5137]<br>toJavaRDD at <unknown>:0"]; } } 5124->5125; 5125->5126; 5126->5127; 5127->5128; 5128->5129; 5129->5130; 5130->5131; 5131->5132; 5132->5133; 5133->5134; 5134->5135; 5135->5136; 5136->5137; 5137->5138; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time