to standardize: when you need zero mean and unit variance for gradient-based optimization

Why do we need to make data uniform before training a model?

Imagine you're trying to teach a robot to recognize different types of fruits. If some fruits are bigger and some are smaller, the robot might struggle to learn correctly.

To help the robot, we adjust all fruit sizes to a standard size. This way, the robot can focus on learning the shapes and colors instead of dealing with varying sizes.

Example

If apples range from 1 to 3 inches in diameter and we standardize them to a 2-inch diameter, the robot sees all apples as the same size.

Remember this

Standardizing data to zero mean and unit variance helps gradient-based optimization algorithms learn more efficiently.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Boosting (machine learning)

Boosting reduces bias in ML models

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

gradient accumulation simulates larger batch sizes without more memory

Can you train a machine like you do with a computer?

to normalize features: when features have different scales and you use distance-based methods

Why do some things need to be adjusted to compare fairly?

Proximal gradient methods for learning

Why can't we always find the best path in a maze?

to use random forests: when you want a strong baseline with minimal hyperparameter tuning

Why not always use the best single tree for predictions?

Swipe through 100 ML concepts daily

Open Pocket Polymath