3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1764 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 462
Associated Job Ids:
1764
DAG Visualization
digraph G { subgraph clusterstage_1764 { label="Stage 1764"; 18526 [labelType="html" label="SQLExecutionRDD [18526]<br>run at <unknown>:0"]; 18534 [labelType="html" label="SQLExecutionRDD [18534]<br>toJavaRDD at <unknown>:0"]; 18536 [labelType="html" label="PythonRDD [18536]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster41459 { label="MicroBatchScan"; 18522 [labelType="html" label="DataSourceRDD [18522]<br>run at <unknown>:0"]; 18523 [labelType="html" label="MapPartitionsRDD [18523]<br>run at <unknown>:0"]; } subgraph cluster41458 { label="Project"; 18524 [labelType="html" label="MapPartitionsRDD [18524]<br>run at <unknown>:0"]; } subgraph cluster41454 { label="WholeStageCodegen (1)"; 18525 [labelType="html" label="MapPartitionsRDD [18525]<br>run at <unknown>:0"]; } subgraph cluster41468 { label="WholeStageCodegen (1)"; 18527 [labelType="html" label="MapPartitionsRDD [18527]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41467 { label="Filter"; 18528 [labelType="html" label="MapPartitionsRDD [18528]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41466 { label="Project"; 18529 [labelType="html" label="MapPartitionsRDD [18529]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41463 { label="WholeStageCodegen (2)"; 18530 [labelType="html" label="MapPartitionsRDD [18530]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41462 { label="Project"; 18531 [labelType="html" label="MapPartitionsRDD [18531]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41461 { label="DeserializeToObject"; 18532 [labelType="html" label="MapPartitionsRDD [18532]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41460 { label="MapPartitions"; 18533 [labelType="html" label="MapPartitionsRDD [18533]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster41470 { label="mapPartitions"; 18535 [labelType="html" label="MapPartitionsRDD [18535]<br>toJavaRDD at <unknown>:0"]; } } 18522->18523; 18523->18524; 18524->18525; 18525->18526; 18526->18527; 18527->18528; 18528->18529; 18529->18530; 18530->18531; 18531->18532; 18532->18533; 18533->18534; 18534->18535; 18535->18536; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time