Resampling (statistics)

Bootstrapping samples with replacement to estimate distributions

Resampling methods, including bootstrapping, involve creating new samples from an observed sample. Bootstrapping specifically uses resampling with replacement to estimate the distribution of a statistic.

Example

From a sample of 100 test scores, we repeatedly draw 100 scores with replacement to create new samples, then calculate the mean for each sample to estimate the distribution of the sample mean.

Remember this

Bootstrapping helps estimate the variability and distribution of a statistic without relying on strong parametric assumptions.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

importance sampling does: reweights samples from proposal to estimate target expectation

Importance sampling estimates target expectations using samples from a different distribution

rejection sampling does: samples from target by accepting/rejecting proposals

Rejection sampling generates observations from a target distribution

Boosting (machine learning)

Boosting reduces bias in ML models

denoising score matching does: learns to denoise, which equals learning the score

Propensity score matching (PSM) reduces bias in treatment effect estimates

GraphSAGE does: samples and aggregates a fixed-size neighborhood

GraphSAGE samples and aggregates a fixed-size neighborhood

Metropolis–Hastings algorithm

Metropolis-Hastings algorithm samples from difficult distributions

Swipe through 100 ML concepts daily

Open Pocket Polymath