Requirements¶
The mandatory requirements to run the Xonai Accelerator are:
Apache Spark 3.1+
Compatible Spark runtimes:
Open-source distribution of Apache Spark 3.1+
AWS EMR
Databricks (experimental)
Dataproc (experimental)
Access to Xonai Accelerator JARs
Supported Spark Runtimes¶
The Xonai Accelerator is distributed as a set of JARs, one for each supported Spark runtime as shown in the following table:
Spark Version |
Runtime |
||||
|---|---|---|---|---|---|
Open Source |
AWS EMR |
Databricks |
Dataproc |
||
3.1.1 |
(6.3.X) |
||||
3.1.2 |
(6.4.0, 6.5.0) |
(9.1 LTS) |
|||
3.1.3 |
(2.0.X) |
||||
3.2.0 |
(6.6.0) |
||||
3.2.1 |
(6.7.0) |
(10.4 LTS) |
|||
3.2.2 |
|||||
3.2.3 |
|||||
3.2.4 |
|||||
3.3.0 |
(6.8.X, 6.9.X) |
(11.3 LTS) |
|||
3.3.1 |
(6.10.X) |
||||
3.3.2 |
(6.11.X) |
(12.2 LTS) |
(2.1.X) |
||
3.3.3 |
|||||
3.3.4 |
|||||
3.4.0 |
(6.12.0) |
||||
3.4.1 |
(6.13.0, 6.14.0, 6.15.0) |
(13.3 LTS) |
|||
3.4.2 |
|||||
3.4.3 |
|||||
3.4.4 |
|||||
3.5.0 |
(7.0.0, 7.1.0) |
(14.3 LTS, 15.4 LTS) |
|||
3.5.1 |
(7.2.0, 7.3.0) |
(2.2.X) |
|||
3.5.2 |
(7.4.0, 7.5.0) |
||||
3.5.3 |
(7.6.0, 7.7.0) |
||||
3.5.4 |
(7.8.0) |
||||
3.5.5 |
|||||
Each JAR uses the following naming convention:
xonai-spark-plugin-<package>-<runtime>-<release>-<platform>.jar
Depending on your execution environment, each tag may be named as listed in the following table:
Package <package> |
Label |
|---|---|
Open-source distribution |
oss |
Amazon EMR |
emr |
Databricks |
dbx |
Google Dataproc |
gdp |
Supported Linux Distributions¶
The Xonai Accelerator works on any glibc-2.17-or-later-based amd64 or arm64 Linux distribution. A non-exhaustive list includes, but is not limited to:
Debian >= 8
Ubuntu >= 14.06
Red Hat Enterprise Linux >= 7
CentOS >= 7
Supported JDKs and Scala¶
The Xonai Accelerator is compatible with JDK 1.8, 11 and 17. JARs may be built with Scala 2.12 and/or 2.13 depending of the Spark version.