Why can't we always use the same samples to figure out what's happening in a complex system?
Image: Spacecoastcreative, CC0, via Wikimedia Commons
importance sampling does: reweights samples from proposal to estimate target expectation
Why can't we always use the same samples to figure out what's happening in a complex system?
Imagine you're trying to figure out the average height of trees in a forest, but you can only measure the height of a few trees easily. You don't want to measure every single tree because it would be too time-consuming and expensive.
Instead of measuring every tree, you decide to measure a bunch of trees that are taller than average and then use that information to guess the average height of all the trees. You're using a shortcut to get closer to the answer without measuring everything.
Example
You measure 10 tall trees and find their average height is 30 feet. Since these trees are taller than average, you infer that the average height of all trees is probably less than 30 feet.
Remember this
Importance sampling lets us estimate the average height of all trees by cleverly choosing a smaller sample that gives us a good approximation.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Model selection
Why do we sometimes choose simpler explanations over complex ones?
Prior probability
Why do we sometimes need to start with an educated guess before diving into new data?
Metropolis–Hastings algorithm
Metropolis-Hastings algorithm samples from difficult distributions
Likelihood function
Why can't we just guess probabilities correctly all the time?
log-probabilities are used instead of probabilities: avoids numerical underflow
Why can't we just add up tiny chances over time?
rejection sampling does: samples from target by accepting/rejecting proposals
How do you randomly pick a number following a strange rule?
Swipe through more Machine Learning concepts
Open Pocket Polymath