Parallelism Specifics

Extension for scikit-learn* supports the n_jobs parameter of the original scikit-learn with the following differences:

• n_jobs parameter is supported for all estimators patched by Extension for scikit-learn*, while scikit-learn enables it for selected estimators only.

• n_jobs estimator parameter sets the number of threads used by the underlying oneAPI Data Analytics Library.

• Extension for scikit-learn* doesn’t use joblib for parallelism in patched estimators and functions.

• The only low-level parallelism library used by Extension for scikit-learn* is oneTBB (through the oneAPI Data Analytics Library and oneMKL).

• If n_jobs is not specified, Extension for scikit-learn* uses all available threads whereas scikit-learn is single-threaded by default. Note that the deprecated daal4py module uses a global configuration instead of per-object n_jobs arguments, with the default also being all available threads.

Hint

If serving estimators from the Extension for scikit-learn* in online services - e.g. through a REST framework like Flask - where requests might be parallelized through a different tool like GUnicorn, one might want to set n_jobs=1 in estimators objects after fitting - e.g.

estimator.fit(...)
estimator.set_params(n_jobs=1)

And one might want to also limit parallelism of the higher-level framework (such as GUnicorn) to number of threads or physical cores in the machine.

See also the Intel guide for optimization of scikit-learn workflows - same considerations apply to estimators from the Extension for scikit-learn*.

Extension for scikit-learn* follows the same rules as scikit-learn for the calculation of the n_jobs parameter value.

When scikit-learn’s utilities with built-in parallelism are used (for example, sklearn.model_selection.GridSearchCV or sklearn.model_selection.VotingClassifier), Extension for scikit-learn* tries to determine the optimal number of threads per job using hints provided by joblib / threadpoolctl. If n_jobs is not specified for underlying estimator(s), Extension for scikit-learn* sets it to the number of available threads (usually the number of logical CPUs divided by n_jobs set for higher-level parallelized entities).

Python threads and thread safety

In general, accelerated computations offered by estimators from the Extension for scikit-learn* do not raise the Python GIL, thus they are not compatible with parallelization backends that rely on Python threads. Instead, process-based parallelism is recommended, which is the default mode in tools like joblib and by extension in metaestimators from scikit-learn. Note that builds of the Extension for scikit-learn* for free-threaded Python are not offered at this moment.

Besides GIL usage in the Extension for scikit-learn*, there are other considerations with concurrent usage in Python threads, even if running under the Python GIL:

• Internally, the number of threads for calls to estimator methods from the Extension for scikit-learn* is managed through global variables - thus, if multiple calls to estimators with different n_jobs are performed in parallel through Python threads, there might be threading races that override one another’s configuration, potentially leading to process-wide crashes.

• The patched classes sklearn.linear_model.LogisticRegression and sklearn.linear_model.LogisticRegressionCV in particular are not suitable for parallel calls in Python threads regardless of the n_jobs parameter due to usage of other Python-level global state variables. Attempting to fit multiple logistic regression estimator objects in parallel might result in crashes and incorrect estimations.

• While most estimators only set their attributes and internal state during calls to .fit() and then use them without modifications in .predict() and similar, estimators based on K-nearest neighbors instead set their attributes / state also during calls to methods like .transform() - hence, when it comes to KNN-based classes, if calling methods on the same estimator instance from multiple threads, they might override one another when setting variables, leading to incorrect results and potential crashes. These classes include:

  • sklearn.neighbors.NearestNeighbors.

  • sklearn.neighbors.KNeighborsRegressor.

  • sklearn.neighbors.KNeighborsClassifier.

  • sklearn.neighbors.LocalOutlierFactor.

Other considerations

Note

Environment variables such as OMP_NUM_THREADS, MKL_NUM_THREADS, OPENBLAS_NUM_THREADS, and others used by low-level parallelism libraries do not affect Extension for scikit-learn*, nor does the mkl-service package.

Note

n_jobs has no effect if computations are performed on GPU.

Note

threadpoolctl context has no effect on Extension for scikit-learn* threading if n_jobs is specified and non-negative. If n_jobs is equal to 0 or not specified then the number from threadpoolctl is propagated to Extension for scikit-learn*. If n_jobs is negative then the threadpoolctl’s number will be max(1, n_threadpoolctl + n_jobs + 1).

Note

Extension for scikit-learn* threading doesn’t automatically avoid nested parallelism when used in conjunction with OpenMP and/or Python threads, unless the Python threads are scheduled through joblib.

Warning

If several instances of Extension for scikit-learn* algorithms are run sequentially and the n_jobs parameter for the first run is significantly greater than for subsequent ones, it may result in performance degradation due to a known issue with oneTBB.

Setting the DEBUG verbosity setting will produce logs indicating when the number of threads used is different from the default (number of logical threads in the machine).