3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1480 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 4
Input Size / Records:
0.0 B / 405
Associated Job Ids:
1480
DAG Visualization
digraph G { subgraph clusterstage_1480 { label="Stage 1480"; 15544 [labelType="html" label="SQLExecutionRDD [15544]<br>run at <unknown>:0"]; 15552 [labelType="html" label="SQLExecutionRDD [15552]<br>toJavaRDD at <unknown>:0"]; 15554 [labelType="html" label="PythonRDD [15554]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster34785 { label="MicroBatchScan"; 15540 [labelType="html" label="DataSourceRDD [15540]<br>run at <unknown>:0"]; 15541 [labelType="html" label="MapPartitionsRDD [15541]<br>run at <unknown>:0"]; } subgraph cluster34784 { label="Project"; 15542 [labelType="html" label="MapPartitionsRDD [15542]<br>run at <unknown>:0"]; } subgraph cluster34780 { label="WholeStageCodegen (1)"; 15543 [labelType="html" label="MapPartitionsRDD [15543]<br>run at <unknown>:0"]; } subgraph cluster34794 { label="WholeStageCodegen (1)"; 15545 [labelType="html" label="MapPartitionsRDD [15545]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34793 { label="Filter"; 15546 [labelType="html" label="MapPartitionsRDD [15546]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34792 { label="Project"; 15547 [labelType="html" label="MapPartitionsRDD [15547]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34789 { label="WholeStageCodegen (2)"; 15548 [labelType="html" label="MapPartitionsRDD [15548]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34788 { label="Project"; 15549 [labelType="html" label="MapPartitionsRDD [15549]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34787 { label="DeserializeToObject"; 15550 [labelType="html" label="MapPartitionsRDD [15550]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34786 { label="MapPartitions"; 15551 [labelType="html" label="MapPartitionsRDD [15551]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster34796 { label="mapPartitions"; 15553 [labelType="html" label="MapPartitionsRDD [15553]<br>toJavaRDD at <unknown>:0"]; } } 15540->15541; 15541->15542; 15542->15543; 15543->15544; 15544->15545; 15545->15546; 15546->15547; 15547->15548; 15548->15549; 15549->15550; 15550->15551; 15551->15552; 15552->15553; 15553->15554; }
Event Timeline
Enable zooming
Tasks: 4. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time