Cluster-Scoped Installation¶

The Xonai Accelerator can be installed on a cluster via an init script that copies the plugin JAR to a location where driver library dependencies are referenced. It is then activated on Spark jobs by adding a few extra properties to Spark configuration.

Supported Types of Compute¶

The Xonai Accelerator can be activated in the following types of compute:

1. All-purpose compute.

2. Job compute.

For more information about Databricks types of compute, please refer to the Databricks compute documentation.

Supported Databricks Runtime Versions¶

The Xonai Accelerator is compatible with the Databricks runtime versions listed as shown in the following table, with the corresponding JAR artifact name for each processor type:

For more information about Databricks runtime releases, please refer to the Databricks release notes.

Create an Init Script¶

To create an init script that install Xonai Accelerator JAR in a Compute cluster:

1. Navigate to Workspace > Home from the left navigation panel.

2. Click Create > File at the top right corner.

3. Enter the file name xonai-init.sh.

4. Copy the following content and change the placeholders as described after:

   1 2	%sh cp /dbfs/xonai/xonai-spark-plugin-dbx-<release>-<version>-stable-linux-<arch>.jar /databricks/jars/

5. Change <runtime>, <release> and <arch> placeholders with:

   1. The Databricks runtime version you are using (e.g. 14.3)

   2. The Xonai JAR release version you want to use

   3. The Worker processor architecture: amd64 for Intel/AMD or arm64 for Graviton/ARM

For more information about cluster-scoped init scripts, please refer to the Databricks script documentation.

Next Steps¶

Please refer to our activation guides to learn how to activate the Xonai Accelerator for the different types of Databricks compute with the init script created in this page.