to normalize features: when features have different scales and you use distance-based methods

Why do some things need to be adjusted to compare fairly?

Imagine you're trying to compare the heights of different plants in a garden, but some plants grow much taller than others.

To make a fair comparison, you need to adjust the heights so they're on the same scale. This way, you can easily see which plants are taller or shorter.

Example

If Plant A is 10 inches tall and Plant B is 30 inches tall, you can think of Plant B as being 3 times taller than Plant A.

Remember this

Normalizing features helps you compare things on the same scale.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

to standardize: when you need zero mean and unit variance for gradient-based optimization

Why do we need to make data uniform before training a model?

L1 vs L2 regularization: L1 gives sparsity (feature selection), L2 gives small weights

L1 regularization: L1 = L2 + sparsity; L2 regularization: L2 = L1 + small weights

Pre-LN

Why is normalizing data like tuning instruments before a concert?

cosine similarity works better than Euclidean distance in high dimensions

Cosine similarity measures orientation, not magnitude, making it more robust to irrelevant dimensions in high-dimensional spaces

cosine similarity is preferred over dot product for normalized embeddings

Why do we need a special way to measure similarity in high-dimensional spaces?

Boosting (machine learning)

Boosting reduces bias in ML models

Swipe through 100 ML concepts daily

Open Pocket Polymath