Hoeffding's inequality bounds tail probability for sums of bounded random variables
Image: CC BY-SA 3.0, via Wikimedia Commons
Hoeffding's inequality
Hoeffding's inequality bounds tail probability for sums of bounded random variables
Hoeffding's inequality offers an upper limit on the probability that the sum of independent random variables strays significantly from its expected value. This inequality was introduced by Wassily Hoeffding in 1963. It is a foundational result in probability theory, providing a way to quantify the concentration of sums of random variables.
Example
Consider a sequence of 100 independent random variables, each uniformly distributed between 0 and 1. Hoeffding's inequality can be used to bound the probability that their sum deviates from the expected value (which is 50) by more than a certain amount.
Remember this
Understanding Hoeffding's inequality helps in assessing the reliability of statistical estimates and in designing algorithms with guaranteed performance bounds.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Markov's inequality
Ever wondered how math can predict unlikely events?
Chernoff bound
Chernoff bounds provide exponentially tight concentration inequalities
Chebyshev's inequality
Chebyshev's inequality limits the probability of deviation from the mean
the Johnson-Lindenstrauss lemma says
Can we shrink big data without losing important details?
List of unsolved problems in mathematics
Why do random points in high dimensions seem to be evenly spaced?
Kolmogorov complexity
Kolmogorov complexity is uncomputable
Swipe through 100 ML concepts daily
Open Pocket Polymath