Why do we sometimes choose simpler explanations over complex ones?
Image: Fgpacini, CC BY-SA 4.0, via Wikimedia Commons
Model selection
Why do we sometimes choose simpler explanations over complex ones?
Imagine you're trying to figure out why your plants are wilting. You have a bunch of possible reasons like too much sun, not enough water, or pests.
You want to find the most likely reason without getting overwhelmed by too many possibilities. It's like picking the simplest explanation that still fits all the clues.
Example
If you notice both your sun-loving flowers and shade-preferring plants are wilting, you might guess it's not just too much sun but possibly not enough water.
Remember this
The idea is to use a simpler explanation that captures all the important clues, known as a "sufficient statistic."
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Sufficient statistic
Sufficiency captures all information about θ in the data
Prior probability
Why do we sometimes need to start with an educated guess before diving into new data?
importance sampling does: reweights samples from proposal to estimate target expectation
Why can't we always use the same samples to figure out what's happening in a complex system?
soft targets carry more information than hard labels: they encode class similarities
Why do some learning methods need to explore more than others?
to normalize features: when features have different scales and you use distance-based methods
Why do some things need to be adjusted to compare fairly?
Bias vs variance: high bias = underfitting, high variance = overfitting
Can a perfect fit to past data predict future events?
Swipe through more Machine Learning concepts
Open Pocket Polymath