LASSO uses L1 to do feature selection by driving coefficients to exactly zero

Why do some numbers disappear when solving complex problems?

Imagine you're trying to balance a scale with weights representing different features of a dataset. Some weights are too heavy and throw off the balance.

LASSO helps by making some weights lighter or even removing them entirely, simplifying the balance without losing the overall picture.

Example

If you have weights of 1, 2, 3, and 4, LASSO might remove the 4, keeping just 1, 2, and 3 to balance the scale more effectively.

Remember this

LASSO uses L1 norm to drive some feature coefficients to zero, simplifying models.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Regularization (mathematics)

L1 regularization results in sparse solutions

L1 vs L2 regularization: L1 gives sparsity (feature selection), L2 gives small weights

L1 regularization: L1 = L2 + sparsity; L2 regularization: L2 = L1 + small weights

Convolutional neural network

Can a neural network learn too well?

Proximal gradient methods for learning

Why can't we always find the best path in a maze?

Ridge regression uses L2 to shrink coefficients without eliminating them

Why do some roads get smoother and straighter over time?

to standardize: when you need zero mean and unit variance for gradient-based optimization

Why do we need to make data uniform before training a model?

Swipe through 100 ML concepts daily

Open Pocket Polymath