Why do we sometimes need to start with an educated guess before diving into new data?
Image: Unknown authorUnknown author, CC BY 4.0, via Wikimedia Commons
Prior probability
Why do we sometimes need to start with an educated guess before diving into new data?
Imagine you're baking a cake and you know from past experience that a certain brand of flour always works well. You don't need to test every new brand before deciding which to buy.
Bayesian estimation is like using your past cake baking experience to make an educated guess about new brands of flour. It combines what you already know (the prior) with new information (the data) to make a better guess (the posterior).
Example
You have a prior belief that Brand A flour is good (80% chance), and you try it in a new cake recipe. You find it works well (new data). Bayesian estimation updates your belief to 85% that Brand A flour is good.
Remember this
Bayesian estimation refines predictions by blending past knowledge with new evidence, unlike frequentist estimation which relies solely on new data.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Bayesian inference
Ever wondered how doctors update diagnoses as new symptoms arise?
Bias vs variance: high bias = underfitting, high variance = overfitting
Can a perfect fit to past data predict future events?
importance sampling does: reweights samples from proposal to estimate target expectation
Why can't we always use the same samples to figure out what's happening in a complex system?
Fisher information
Fisher information measures information about unknown parameters
the back-door criterion identifies: sufficient adjustment sets for causal estimation
Can we trust studies without random experiments?
MoE models have more parameters but similar compute cost
MoE models distribute parameters across k experts, reducing active experts' compute cost
Swipe through 100 ML concepts daily
Open Pocket Polymath