Metrics¶
This page documents FuseCore-specific metrics that are displayed for SQL queries in the Spark UI and in the History Server.
WholeStageCodegen¶
Metrics are presented in a XonStats node.
Metric |
Description |
||
|---|---|---|---|
max input batch size |
The maximum number of rows of input batches per task |
||
max output batch size |
The maximum number of rows of output batches per task |
||
peak memory size total |
The peak memory size used by tasks |
||
prepare time total |
The time spent preparing to process batches of data, such as compiling and program invocation overheads |
||
processing time total |
The time spent processing batches of data by FuseCore JIT-compiled programs |
||
spill size total |
Total amount of data that was spilled from memory to disk |
||
XonExchange¶
Metric |
Description |
||
|---|---|---|---|
data read time total |
The time spent reading data from the shuffle |
||
data write time total |
The time spent writing data to the shuffle |
||
repartition time total |
The time spent repartitioning batches |
||
repartition spill size total |
The size of the data spilled to the disk to repartition batches |
||
repartition peak memory size total |
The peak memory size used to repartition batches |
||
XonParquetScan¶
Metric |
Description |
||
|---|---|---|---|
init reader time total |
The time spent computing information to be used by the Parquet reader |
||
page read time total |
The time spent reading Parquet pages into memory |
||
batch read time total |
The time spent transforming Parquet pages into data batches |
||