3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1461 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 359
Associated Job Ids:
1461
DAG Visualization
digraph G { subgraph clusterstage_1461 { label="Stage 1461"; 15334 [labelType="html" label="SQLExecutionRDD [15334]<br>run at <unknown>:0"]; 15348 [labelType="html" label="SQLExecutionRDD [15348]<br>toJavaRDD at <unknown>:0"]; 15350 [labelType="html" label="PythonRDD [15350]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster34315 { label="MicroBatchScan"; 15330 [labelType="html" label="DataSourceRDD [15330]<br>run at <unknown>:0"]; 15331 [labelType="html" label="MapPartitionsRDD [15331]<br>run at <unknown>:0"]; } subgraph cluster34314 { label="Project"; 15332 [labelType="html" label="MapPartitionsRDD [15332]<br>run at <unknown>:0"]; } subgraph cluster34310 { label="WholeStageCodegen (1)"; 15333 [labelType="html" label="MapPartitionsRDD [15333]<br>run at <unknown>:0"]; } subgraph cluster34332 { label="WholeStageCodegen (1)"; 15341 [labelType="html" label="MapPartitionsRDD [15341]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34331 { label="Filter"; 15342 [labelType="html" label="MapPartitionsRDD [15342]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34330 { label="Project"; 15343 [labelType="html" label="MapPartitionsRDD [15343]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34327 { label="WholeStageCodegen (2)"; 15344 [labelType="html" label="MapPartitionsRDD [15344]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34326 { label="Project"; 15345 [labelType="html" label="MapPartitionsRDD [15345]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34325 { label="DeserializeToObject"; 15346 [labelType="html" label="MapPartitionsRDD [15346]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34324 { label="MapPartitions"; 15347 [labelType="html" label="MapPartitionsRDD [15347]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34334 { label="mapPartitions"; 15349 [labelType="html" label="MapPartitionsRDD [15349]<br>toJavaRDD at <unknown>:0"]; } } 15330->15331; 15331->15332; 15332->15333; 15333->15334; 15334->15341; 15341->15342; 15342->15343; 15343->15344; 15344->15345; 15345->15346; 15346->15347; 15347->15348; 15348->15349; 15349->15350; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time