3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1508 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 413
Associated Job Ids:
1508
DAG Visualization
digraph G { subgraph clusterstage_1508 { label="Stage 1508"; 15838 [labelType="html" label="SQLExecutionRDD [15838]<br>run at <unknown>:0"]; 15846 [labelType="html" label="SQLExecutionRDD [15846]<br>toJavaRDD at <unknown>:0"]; 15848 [labelType="html" label="PythonRDD [15848]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster35443 { label="MicroBatchScan"; 15834 [labelType="html" label="DataSourceRDD [15834]<br>run at <unknown>:0"]; 15835 [labelType="html" label="MapPartitionsRDD [15835]<br>run at <unknown>:0"]; } subgraph cluster35442 { label="Project"; 15836 [labelType="html" label="MapPartitionsRDD [15836]<br>run at <unknown>:0"]; } subgraph cluster35438 { label="WholeStageCodegen (1)"; 15837 [labelType="html" label="MapPartitionsRDD [15837]<br>run at <unknown>:0"]; } subgraph cluster35452 { label="WholeStageCodegen (1)"; 15839 [labelType="html" label="MapPartitionsRDD [15839]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35451 { label="Filter"; 15840 [labelType="html" label="MapPartitionsRDD [15840]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35450 { label="Project"; 15841 [labelType="html" label="MapPartitionsRDD [15841]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35447 { label="WholeStageCodegen (2)"; 15842 [labelType="html" label="MapPartitionsRDD [15842]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35446 { label="Project"; 15843 [labelType="html" label="MapPartitionsRDD [15843]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35445 { label="DeserializeToObject"; 15844 [labelType="html" label="MapPartitionsRDD [15844]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35444 { label="MapPartitions"; 15845 [labelType="html" label="MapPartitionsRDD [15845]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster35454 { label="mapPartitions"; 15847 [labelType="html" label="MapPartitionsRDD [15847]<br>toJavaRDD at <unknown>:0"]; } } 15834->15835; 15835->15836; 15836->15837; 15837->15838; 15838->15839; 15839->15840; 15840->15841; 15841->15842; 15842->15843; 15843->15844; 15844->15845; 15845->15846; 15846->15847; 15847->15848; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time