Distributed Mode

Intel® Extension for Scikit-learn* offers Single Program, Multiple Data (SPMD) supported interfaces for distributed computing. Several GPU-supported algorithms also provide distributed, multi-GPU computing capabilities via integration with mpi4py. The prerequisites match those of GPU computing, along with an MPI backend of your choice (Intel MPI recommended, available via impi-devel python package) and the mpi4py python package. If using Intel® Extension for Scikit-learn* installed from sources, ensure that the spmd_backend is built.

Note that Intel® Extension for Scikit-learn* now supports GPU offloading to speed up MPI operations. This is supported automatically with some MPI backends, but in order to use GPU offloading with Intel MPI, set the following environment variable (providing data on device without this may lead to a runtime error):

export I_MPI_OFFLOAD=1

Estimators can be imported from the sklearnex.spmd module. Data should be distributed across multiple nodes as desired, and should be transfered to a dpctl or dpnp array before being passed to the estimator. View a full example of this process in the Intel® Extension for Scikit-learn* repository, where many examples of our SPMD-supported estimators are available: https://github.com/intel/scikit-learn-intelex/blob/main/examples/sklearnex/. To run:

mpirun -n 4 python linear_regression_spmd.py

Note that additional mpirun arguments can be added as desired. SPMD-supported estimators are listed in the algorithms support documentation.

Additionally, daal4py offers some distributed functionality, see documentation for further details.