Diffusion model

q(x_t|x_{t-1}) adds Gaussian noise at each step

The forward diffusion process in diffusion models involves adding Gaussian noise incrementally to transform data into a simpler distribution. This process gradually corrupts the original data, making it easier to learn the reverse process during training. The noise addition at each step follows a Markov chain, ensuring that each step depends only on the previous one.

Example

Consider an image initially clean. During the forward diffusion process, Gaussian noise is added at each time step, gradually transforming the image into a noisy, blurred version. This transformation follows a Markov chain, where each step's noise addition depends solely on the image's state at the previous step.

Remember this

Understanding the forward diffusion process is crucial for grasping how diffusion models learn to generate new data samples by reversing the noise addition process. This knowledge is fundamental for developing and improving generative models in machine learning.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Langevin dynamics does: adds noise to gradient descent to sample from a distribution

Langevin dynamics uses stochastic differential equations

Stable Diffusion

Stable Diffusion generates images from text descriptions

the momentum term does: v_t = βv_{t-1} + ∇L, accumulates gradient direction

Momentum term accelerates convergence in the gradient direction

the reverse process learns: p_θ(x_{t-1}|x_t)

Ancestral reconstruction extrapolates characteristics back in time

Markov chain Monte Carlo

MCMC samples from complex posterior distributions

Vanishing gradient problem

Residual connections help by allowing gradient flow through the skip connection

Swipe through 100 ML concepts daily

Open Pocket Polymath