Quick Activation

This page documents the steps to activate the Xonai Accelerator for the open-source distribution of Apache Spark.

Submit a Spark Job with Xonai

The Xonai Accelerator can be activated on a per-application basis via Spark 3 configuration just by adding the following properties:

--jars <scheme>://<path>/xonai-spark-plugin-<distro>-<version>-<arch>.jar
--conf spark.plugins=com.xonai.spark.SQLPlugin

Then, you need to configure executor memory properties according to this guide.

Activating the Xonai Shuffle Manager

Assuming you meet the prerequisites to use the Xonai shuffle manager, you need to add the following properties:

--conf spark.shuffle.manager=com.xonai.spark.ShuffleManager
--conf spark.driver.extraClassPath=<local-path-to-xonai-accelerator-jar>
--conf spark.executor.extraClassPath=<local-path-to-xonai-accelerator-jar>

The spark.*.extraClassPath properties above need to be appended if already specified.