.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/linear_model/plot_logistic_l1_l2_sparsity.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code or to run this example in your browser via JupyterLite or Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_linear_model_plot_logistic_l1_l2_sparsity.py: ============================================== L1 Penalty and Sparsity in Logistic Regression ============================================== Comparison of the sparsity (percentage of zero coefficients) of solutions when L1, L2 and Elastic-Net penalty are used for different values of C. We can see that large values of C give more freedom to the model. Conversely, smaller values of C constrain the model more. In the L1 penalty case, this leads to sparser solutions. As expected, the Elastic-Net penalty sparsity is between that of L1 and L2. We classify 8x8 images of digits into two classes: 0-4 against 5-9. The visualization shows coefficients of the models for varying C. .. GENERATED FROM PYTHON SOURCE LINES 17-93 .. image-sg:: /auto_examples/linear_model/images/sphx_glr_plot_logistic_l1_l2_sparsity_001.png :alt: L1 penalty, Elastic-Net l1_ratio = 0.5, L2 penalty :srcset: /auto_examples/linear_model/images/sphx_glr_plot_logistic_l1_l2_sparsity_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none C=1.00 Sparsity with L1 penalty: 4.69% Sparsity with Elastic-Net penalty: 4.69% Sparsity with L2 penalty: 4.69% Score with L1 penalty: 0.90 Score with Elastic-Net penalty: 0.90 Score with L2 penalty: 0.90 C=0.10 Sparsity with L1 penalty: 29.69% Sparsity with Elastic-Net penalty: 14.06% Sparsity with L2 penalty: 4.69% Score with L1 penalty: 0.90 Score with Elastic-Net penalty: 0.90 Score with L2 penalty: 0.90 C=0.01 Sparsity with L1 penalty: 84.38% Sparsity with Elastic-Net penalty: 68.75% Sparsity with L2 penalty: 4.69% Score with L1 penalty: 0.86 Score with Elastic-Net penalty: 0.88 Score with L2 penalty: 0.89 | .. code-block:: Python # Authors: Alexandre Gramfort # Mathieu Blondel # Andreas Mueller # License: BSD 3 clause import matplotlib.pyplot as plt import numpy as np from sklearn import datasets from sklearn.linear_model import LogisticRegression from sklearn.preprocessing import StandardScaler X, y = datasets.load_digits(return_X_y=True) X = StandardScaler().fit_transform(X) # classify small against large digits y = (y > 4).astype(int) l1_ratio = 0.5 # L1 weight in the Elastic-Net regularization fig, axes = plt.subplots(3, 3) # Set regularization parameter for i, (C, axes_row) in enumerate(zip((1, 0.1, 0.01), axes)): # Increase tolerance for short training time clf_l1_LR = LogisticRegression(C=C, penalty="l1", tol=0.01, solver="saga") clf_l2_LR = LogisticRegression(C=C, penalty="l2", tol=0.01, solver="saga") clf_en_LR = LogisticRegression( C=C, penalty="elasticnet", solver="saga", l1_ratio=l1_ratio, tol=0.01 ) clf_l1_LR.fit(X, y) clf_l2_LR.fit(X, y) clf_en_LR.fit(X, y) coef_l1_LR = clf_l1_LR.coef_.ravel() coef_l2_LR = clf_l2_LR.coef_.ravel() coef_en_LR = clf_en_LR.coef_.ravel() # coef_l1_LR contains zeros due to the # L1 sparsity inducing norm sparsity_l1_LR = np.mean(coef_l1_LR == 0) * 100 sparsity_l2_LR = np.mean(coef_l2_LR == 0) * 100 sparsity_en_LR = np.mean(coef_en_LR == 0) * 100 print("C=%.2f" % C) print("{:<40} {:.2f}%".format("Sparsity with L1 penalty:", sparsity_l1_LR)) print("{:<40} {:.2f}%".format("Sparsity with Elastic-Net penalty:", sparsity_en_LR)) print("{:<40} {:.2f}%".format("Sparsity with L2 penalty:", sparsity_l2_LR)) print("{:<40} {:.2f}".format("Score with L1 penalty:", clf_l1_LR.score(X, y))) print( "{:<40} {:.2f}".format("Score with Elastic-Net penalty:", clf_en_LR.score(X, y)) ) print("{:<40} {:.2f}".format("Score with L2 penalty:", clf_l2_LR.score(X, y))) if i == 0: axes_row[0].set_title("L1 penalty") axes_row[1].set_title("Elastic-Net\nl1_ratio = %s" % l1_ratio) axes_row[2].set_title("L2 penalty") for ax, coefs in zip(axes_row, [coef_l1_LR, coef_en_LR, coef_l2_LR]): ax.imshow( np.abs(coefs.reshape(8, 8)), interpolation="nearest", cmap="binary", vmax=1, vmin=0, ) ax.set_xticks(()) ax.set_yticks(()) axes_row[0].set_ylabel("C = %s" % C) plt.show() .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.507 seconds) .. _sphx_glr_download_auto_examples_linear_model_plot_logistic_l1_l2_sparsity.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/main?urlpath=lab/tree/notebooks/auto_examples/linear_model/plot_logistic_l1_l2_sparsity.ipynb :alt: Launch binder :width: 150 px .. container:: lite-badge .. image:: images/jupyterlite_badge_logo.svg :target: ../../lite/lab/?path=auto_examples/linear_model/plot_logistic_l1_l2_sparsity.ipynb :alt: Launch JupyterLite :width: 150 px .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_logistic_l1_l2_sparsity.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_logistic_l1_l2_sparsity.py ` .. include:: plot_logistic_l1_l2_sparsity.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_