Memory Modes¶

The Xonai Accelerator uses off-heap memory to process data and it can operate in one of three modes: off-heap, overhead, and dynamic.

The default memory mode is selected based on existing properties and can be overridden with spark.xonai.memory.mode. The amount of memory Xonai can use is also automatically computed and can be overridden with spark.xonai.memory.size.

Off-Heap¶

Xonai uses the Spark off-heap management to acquire memory meaning that Spark and Xonai share a memory pool. This mode is selected by default if spark.memory.offHeap.size and spark.memory.offHeap.enabled are specified.

The default usage looks as the following:

--conf spark.memory.offHeap.size=30g
--conf spark.memory.offHeap.enabled=true

Overhead¶

Xonai uses part of the executor memory overhead with a memory pool separate from Spark. This mode is selected by default if spark.executor.memoryOverhead is specified. The default executor memory overhead is 10% of spark.executor.memory (18% for EMR).

For Xonai to run optimally, the executor memory (JVM) must be reduced and the memory overhead increased. For example, if an application has the following memory configuration:

--conf spark.executor.memory=30g
--conf spark.executor.memoryOverhead=5g

It can be modified to:

--conf spark.executor.memory=10g
--conf spark.executor.memoryOverhead=25g

By default, the Xonai memory size is around overhead - memory * overhead factor with a default factor of 0.2. For the example above the result is spark.xonai.memory.size=23g.

The overhead memory mode is recommended when most or all operations in an application are supported by Xonai. Otherwise, non-supported operations may underperform due to limited memory.

Dynamic¶

Xonai monitors memory allocation by the JVM and it uses non-allocated memory. This mode is selected by default if no other mode is selected. As the JVM grows in size the Xonai memory size decreases.

This memory mode can be configured using:

--conf spark.xonai.memory.mode=dynamic
--conf spark.xonai.memory.size=1g

In this mode, the value of spark.xonai.memory.size defines a lower-bound and defaults to 50% of the executor memory overhead. If the dynamic memory size is low the plugin will stop to offload operations to the Xonai engine.

The dynamic memory mode is the most conservative and it is recommended when testing the plugin for the first time or when using the plugin in applications with frequent code changes.