3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1042 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 436
Associated Job Ids:
1043
DAG Visualization
digraph G { subgraph clusterstage_1042 { label="Stage 1042"; 10946 [labelType="html" label="SQLExecutionRDD [10946]<br>run at <unknown>:0"]; 10960 [labelType="html" label="SQLExecutionRDD [10960]<br>toJavaRDD at <unknown>:0"]; 10962 [labelType="html" label="PythonRDD [10962]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster24488 { label="MicroBatchScan"; 10942 [labelType="html" label="DataSourceRDD [10942]<br>run at <unknown>:0"]; 10943 [labelType="html" label="MapPartitionsRDD [10943]<br>run at <unknown>:0"]; } subgraph cluster24487 { label="Project"; 10944 [labelType="html" label="MapPartitionsRDD [10944]<br>run at <unknown>:0"]; } subgraph cluster24483 { label="WholeStageCodegen (1)"; 10945 [labelType="html" label="MapPartitionsRDD [10945]<br>run at <unknown>:0"]; } subgraph cluster24505 { label="WholeStageCodegen (1)"; 10953 [labelType="html" label="MapPartitionsRDD [10953]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24504 { label="Filter"; 10954 [labelType="html" label="MapPartitionsRDD [10954]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24503 { label="Project"; 10955 [labelType="html" label="MapPartitionsRDD [10955]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24500 { label="WholeStageCodegen (2)"; 10956 [labelType="html" label="MapPartitionsRDD [10956]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24499 { label="Project"; 10957 [labelType="html" label="MapPartitionsRDD [10957]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24498 { label="DeserializeToObject"; 10958 [labelType="html" label="MapPartitionsRDD [10958]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24497 { label="MapPartitions"; 10959 [labelType="html" label="MapPartitionsRDD [10959]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster24507 { label="mapPartitions"; 10961 [labelType="html" label="MapPartitionsRDD [10961]<br>toJavaRDD at <unknown>:0"]; } } 10942->10943; 10943->10944; 10944->10945; 10945->10946; 10946->10953; 10953->10954; 10954->10955; 10955->10956; 10956->10957; 10957->10958; 10958->10959; 10959->10960; 10960->10961; 10961->10962; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time