{ "cells": [ { "cell_type": "markdown", "id": "3768ec43", "metadata": {}, "source": [ "# Intel® Extension for Scikit-learn Ridge Regression for Airlines DepDelay dataset" ] }, { "cell_type": "code", "execution_count": 1, "id": "b1b922d1", "metadata": {}, "outputs": [], "source": [ "from timeit import default_timer as timer\n", "from sklearn import metrics\n", "from sklearn.model_selection import train_test_split\n", "import warnings\n", "from sklearn.datasets import fetch_openml\n", "from sklearn.preprocessing import LabelEncoder\n", "from IPython.display import HTML\n", "\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "markdown", "id": "2a1a9234", "metadata": {}, "source": [ "### Download the data" ] }, { "cell_type": "code", "execution_count": 2, "id": "7e73dc65", "metadata": {}, "outputs": [], "source": [ "x, y = fetch_openml(name=\"Airlines_DepDelay_10M\", return_X_y=True)" ] }, { "cell_type": "markdown", "id": "f852cad8", "metadata": {}, "source": [ "### Preprocessing\n", "Let's encode categorical features with LabelEncoder" ] }, { "cell_type": "code", "execution_count": 3, "id": "27ebb377", "metadata": {}, "outputs": [], "source": [ "for col in [\"UniqueCarrier\", \"Origin\", \"Dest\"]:\n", " le = LabelEncoder().fit(x[col])\n", " x[col] = le.transform(x[col])" ] }, { "cell_type": "markdown", "id": "147b3e82", "metadata": {}, "source": [ "Split the data into train and test sets" ] }, { "cell_type": "code", "execution_count": 4, "id": "0d332789", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((9000000, 9), (1000000, 9), (9000000,), (1000000,))" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.1, random_state=0)\n", "x_train.shape, x_test.shape, y_train.shape, y_test.shape" ] }, { "cell_type": "markdown", "id": "246f819f", "metadata": {}, "source": [ "Normalize the data" ] }, { "cell_type": "code", "execution_count": 5, "id": "454a341c", "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import MinMaxScaler, StandardScaler\n", "\n", "scaler_x = MinMaxScaler()\n", "scaler_y = StandardScaler()" ] }, { "cell_type": "code", "execution_count": 6, "id": "df400504", "metadata": {}, "outputs": [], "source": [ "y_train = y_train.to_numpy().reshape(-1, 1)\n", "y_test = y_test.to_numpy().reshape(-1, 1)\n", "\n", "scaler_x.fit(x_train)\n", "x_train = scaler_x.transform(x_train)\n", "x_test = scaler_x.transform(x_test)\n", "\n", "scaler_y.fit(y_train)\n", "y_train = scaler_y.transform(y_train).ravel()\n", "y_test = scaler_y.transform(y_test).ravel()" ] }, { "cell_type": "markdown", "id": "fe1d4fac", "metadata": {}, "source": [ "### Patch original Scikit-learn with Intel® Extension for Scikit-learn\n", "Intel® Extension for Scikit-learn (previously known as daal4py) contains drop-in replacement functionality for the stock Scikit-learn package. You can take advantage of the performance optimizations of Intel® Extension for Scikit-learn by adding just two lines of code before the usual Scikit-learn imports:" ] }, { "cell_type": "code", "execution_count": 7, "id": "ef6938df", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Intel(R) Extension for Scikit-learn* enabled (https://github.com/uxlfoundation/scikit-learn-intelex)\n" ] } ], "source": [ "from sklearnex import patch_sklearn\n", "\n", "patch_sklearn()" ] }, { "cell_type": "markdown", "id": "20c5ab48", "metadata": {}, "source": [ "Intel® Extension for Scikit-learn patching affects performance of specific Scikit-learn functionality. Refer to the [list of supported algorithms and parameters](https://uxlfoundation.github.io/scikit-learn-intelex/latest/algorithms.html) for details. In cases when unsupported parameters are used, the package fallbacks into original Scikit-learn. If the patching does not cover your scenarios, [submit an issue on GitHub](https://github.com/uxlfoundation/scikit-learn-intelex/issues)." ] }, { "cell_type": "markdown", "id": "f80273e7", "metadata": {}, "source": [ "Training of the Ridge Regression algorithm with Intel® Extension for Scikit-learn for Airlines DepDelay dataset" ] }, { "cell_type": "code", "execution_count": 8, "id": "a4dd1c7e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Intel® extension for Scikit-learn time: 0.06 s'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.linear_model import Ridge\n", "\n", "params = {\n", " \"alpha\": 0.3,\n", " \"fit_intercept\": False,\n", " \"random_state\": 0,\n", " \"copy_X\": False,\n", "}\n", "start = timer()\n", "model = Ridge(random_state=0).fit(x_train, y_train)\n", "train_patched = timer() - start\n", "f\"Intel® extension for Scikit-learn time: {train_patched:.2f} s\"" ] }, { "cell_type": "markdown", "id": "f10b51fc", "metadata": {}, "source": [ "Predict and get a result of the Ridge Regression algorithm with Intel® Extension for Scikit-learn" ] }, { "cell_type": "code", "execution_count": 9, "id": "d4295a26", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Patched Scikit-learn MSE: 1.0014288520708046'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_predict = model.predict(x_test)\n", "mse_metric_opt = metrics.mean_squared_error(y_test, y_predict)\n", "f\"Patched Scikit-learn MSE: {mse_metric_opt}\"" ] }, { "cell_type": "markdown", "id": "cbe6db0d", "metadata": {}, "source": [ "### Train the same algorithm with original Scikit-learn\n", "In order to cancel optimizations, we use *unpatch_sklearn* and reimport the class Ridge" ] }, { "cell_type": "code", "execution_count": 10, "id": "6f64ba97", "metadata": {}, "outputs": [], "source": [ "from sklearnex import unpatch_sklearn\n", "\n", "unpatch_sklearn()" ] }, { "cell_type": "markdown", "id": "f242c6da", "metadata": {}, "source": [ "Training of the Ridge Regression algorithm with original Scikit-learn library for Airlines DepDelay dataset" ] }, { "cell_type": "code", "execution_count": 11, "id": "67243849", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Original Scikit-learn time: 0.70 s'" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.linear_model import Ridge\n", "\n", "start = timer()\n", "model = Ridge(random_state=0).fit(x_train, y_train)\n", "train_unpatched = timer() - start\n", "f\"Original Scikit-learn time: {train_unpatched:.2f} s\"" ] }, { "cell_type": "markdown", "id": "c85a125c", "metadata": {}, "source": [ "Predict and get a result of the Ridge Regression algorithm with original Scikit-learn" ] }, { "cell_type": "code", "execution_count": 12, "id": "cd9e726c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Original Scikit-learn MSE: 1.0014288520708057'" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_predict = model.predict(x_test)\n", "mse_metric_original = metrics.mean_squared_error(y_test, y_predict)\n", "f\"Original Scikit-learn MSE: {mse_metric_original}\"" ] }, { "cell_type": "code", "execution_count": 13, "id": "1bde360d", "metadata": {}, "outputs": [ { "data": { "text/html": [ "

Compare MSE metric of patched Scikit-learn and original

MSE metric of patched Scikit-learn: 1.0014288520708046
MSE metric of unpatched Scikit-learn: 1.0014288520708057
Metrics ratio: 0.9999999999999989

With Scikit-learn-intelex patching you can:

" ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HTML(\n", " f\"

Compare MSE metric of patched Scikit-learn and original

\"\n", " f\"MSE metric of patched Scikit-learn: {mse_metric_opt}
\"\n", " f\"MSE metric of unpatched Scikit-learn: {mse_metric_original}
\"\n", " f\"Metrics ratio: {mse_metric_opt/mse_metric_original}
\"\n", " f\"

With Scikit-learn-intelex patching you can:

\"\n", " f\"\"\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 5 }