Maximum likelihood estimation

How do we find the best-fitting parameters for a model?

Imagine you're trying to fit a straight line to a scatter of points on a graph, like predicting a student's test score based on hours studied.

Think of it as adjusting the line's slope and y-intercept until the line represents the most likely path through all the data points. This process is called maximum likelihood estimation (MLE).

Example

If you have points (1,2), (2,3), and (3,5), MLE helps find the line that best fits these points, minimizing the distance between the line and the actual data points.

Remember this

MLE finds the parameters that make the observed data most probable under the assumed model.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Maximum a posteriori estimation

Maximum a posteriori (MAP) estimate maximizes the posterior density

Expectation–maximization algorithm

EM algorithm iteratively maximizes likelihood estimates with latent variables

Akaike information criterion

How do we choose the best model for our data?

Minimum-variance unbiased estimator

MVUE achieves lower variance than any other unbiased estimator

Metropolis–Hastings algorithm

Metropolis-Hastings algorithm samples from difficult distributions

Robust statistics

How can we estimate the center and spread of data even when it's messy?

Swipe through more Machine Learning concepts

Open Pocket Polymath