Bias vs variance: high bias = underfitting, high variance = overfitting

Can a perfect fit to past data predict future events?

Imagine you're trying to predict the weather for your picnic next week. You've noticed that when you used a simple weather app, it often got it wrong, especially for unusual weather patterns.

You need a balance. A simple app might not catch all the details (high bias), while a super complex app might just guess wildly (high variance). The key is finding the sweet spot.

Example

Your simple app predicted sunny (high bias), but the complex app predicted rain (high variance) for your picnic.

Remember this

The sweet spot is where your predictions are accurate without being too simple or too complex.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Overfitting

Why does fitting too much detail to data ruin predictions?

log-loss / cross-entropy loss penalizes: confident wrong predictions more heavily

How wrong predictions hurt more than they're supposed to

Boosting (machine learning)

Boosting reduces bias in ML models

Prior probability

Why do we sometimes need to start with an educated guess before diving into new data?

Model selection

Why do we sometimes choose simpler explanations over complex ones?

Adam has bias correction: divides by (1-β^t) in early steps

Why do we sometimes need to fix mistakes in computer decisions?

Swipe through more Machine Learning concepts

Open Pocket Polymath