EM algorithm iteratively maximizes likelihood estimates with latent variables
Expectation–maximization algorithm
EM algorithm iteratively maximizes likelihood estimates with latent variables
The EM algorithm is designed to handle statistical models that include unobserved latent variables. It iteratively refines estimates of model parameters by alternating between expectation (E) and maximization (M) steps.
Example
In estimating a mixture of Gaussians, the EM algorithm alternates between estimating the parameters of each Gaussian component (M step) and updating the posterior distribution of the latent variables (E step).
Remember this
Understanding the EM algorithm's iterative process is crucial for effectively estimating parameters in complex models with latent variables.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Fisher information
Fisher information measures information about unknown parameters
Minimum-variance unbiased estimator
MVUE achieves lower variance than any other unbiased estimator
Boosting (machine learning)
Boosting reduces bias in ML models
Metropolis–Hastings algorithm
Metropolis-Hastings algorithm samples from difficult distributions
approximation algorithms guarantee: solution within factor α of optimal
Can we always find the best solution quickly?
to standardize: when you need zero mean and unit variance for gradient-based optimization
Why do we need to make data uniform before training a model?
Swipe through 100 ML concepts daily
Open Pocket Polymath