L1 regularization results in sparse solutions
Regularization (mathematics)
L1 regularization results in sparse solutions
L1 regularization, also known as Lasso, adds a penalty equal to the absolute value of the magnitude of coefficients to the loss function. This penalty encourages the coefficients to be zero, leading to sparse solutions where only a subset of features contributes significantly to the model.
Example
In a linear regression model with L1 regularization, if there are 10 features and the regularization parameter is high, the model may end up with only 3 non-zero coefficients, effectively selecting only 3 features out of the 10.
Remember this
Sparse solutions are beneficial for model interpretability and can lead to better generalization by reducing the risk of overfitting.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Ordinary least squares
OLS minimizes squared differences
Ridge regression
Why do straight lines sometimes fail to fit our data perfectly?
LASSO uses L1 to do feature selection by driving coefficients to exactly zero
Why do some numbers disappear when solving complex problems?
L1 vs L2 regularization: L1 gives sparsity (feature selection), L2 gives small weights
L1 regularization: L1 = L2 + sparsity; L2 regularization: L2 = L1 + small weights
Proximal gradient methods for learning
Why can't we always find the best path in a maze?
Deep learning
Why do smart phones sometimes struggle to recognize new faces?
Swipe through more Machine Learning concepts
Open Pocket Polymath