3.5.8
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
processing-stream-kafka-raw-to-m...
application UI
Details for Stage 1997 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 3
Input Size / Records:
0.0 B / 371
Associated Job Ids:
1998
DAG Visualization
digraph G { subgraph clusterstage_1997 { label="Stage 1997"; 20984 [labelType="html" label="SQLExecutionRDD [20984]<br>run at <unknown>:0"]; 20992 [labelType="html" label="SQLExecutionRDD [20992]<br>toJavaRDD at <unknown>:0"]; 20994 [labelType="html" label="PythonRDD [20994]<br>call at /opt/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py:617"]; subgraph cluster46954 { label="MicroBatchScan"; 20980 [labelType="html" label="DataSourceRDD [20980]<br>run at <unknown>:0"]; 20981 [labelType="html" label="MapPartitionsRDD [20981]<br>run at <unknown>:0"]; } subgraph cluster46953 { label="Project"; 20982 [labelType="html" label="MapPartitionsRDD [20982]<br>run at <unknown>:0"]; } subgraph cluster46949 { label="WholeStageCodegen (1)"; 20983 [labelType="html" label="MapPartitionsRDD [20983]<br>run at <unknown>:0"]; } subgraph cluster46963 { label="WholeStageCodegen (1)"; 20985 [labelType="html" label="MapPartitionsRDD [20985]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46962 { label="Filter"; 20986 [labelType="html" label="MapPartitionsRDD [20986]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46961 { label="Project"; 20987 [labelType="html" label="MapPartitionsRDD [20987]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46958 { label="WholeStageCodegen (2)"; 20988 [labelType="html" label="MapPartitionsRDD [20988]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46957 { label="Project"; 20989 [labelType="html" label="MapPartitionsRDD [20989]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46956 { label="DeserializeToObject"; 20990 [labelType="html" label="MapPartitionsRDD [20990]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46955 { label="MapPartitions"; 20991 [labelType="html" label="MapPartitionsRDD [20991]<br>toJavaRDD at <unknown>:0"]; } subgraph cluster46965 { label="mapPartitions"; 20993 [labelType="html" label="MapPartitionsRDD [20993]<br>toJavaRDD at <unknown>:0"]; } } 20980->20981; 20981->20982; 20982->20983; 20983->20984; 20984->20985; 20985->20986; 20986->20987; 20987->20988; 20988->20989; 20989->20990; 20990->20991; 20991->20992; 20992->20993; 20993->20994; }
Event Timeline
Enable zooming
Tasks: 3. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time