Resampling (statistics)

Bootstrapping samples with replacement to estimate distributions

Resampling methods, including bootstrapping, involve creating new samples from an observed sample. Bootstrapping specifically uses resampling with replacement to estimate the distribution of a statistic.

Example

From a sample of 100 test scores, we repeatedly draw 100 scores with replacement to create new samples, then calculate the mean for each sample to estimate the distribution of the sample mean.

Remember this

Bootstrapping helps estimate the variability and distribution of a statistic without relying on strong parametric assumptions.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Boosting (machine learning)

Boosting reduces bias in ML models

GraphSAGE does: samples and aggregates a fixed-size neighborhood

GraphSAGE samples and aggregates a fixed-size neighborhood

Metropolis–Hastings algorithm

Metropolis-Hastings algorithm samples from difficult distributions

score matching does: learns the gradient of the log-density without normalizing

Ever wonder how we can compare apples and oranges fairly in studies?

importance sampling does: reweights samples from proposal to estimate target expectation

Why can't we always use the same samples to figure out what's happening in a complex system?

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

Swipe through 100 ML concepts daily

Open Pocket Polymath