the reverse process learns: p_θ(x_{t-1}|x_t)

Can we trace back the roots of life?

Imagine trying to figure out the original recipe for a famous dish that has evolved over time, with many cooks adding their own twists.

By looking at the final dish and understanding how each cook's version differs, we can guess what the original recipe might have tasted like. This process of guessing is like working backward in time to find the original state.

Example

If a modern dish has 5 ingredients, and we know that each cook added 1 ingredient over time, we can work backward to guess that the original dish had 4 ingredients.

Remember this

The reverse process helps us understand the original state by examining changes step by step.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Stable Diffusion

Stable Diffusion generates images from text descriptions

t-SNE preserves local structure

Can we see the hidden patterns in a cloud of data points?

Proximal gradient methods for learning

Why can't we always find the best path in a maze?

LoRA (machine learning)

LoRA uses r << d for efficient adaptation

cross-entropy equals negative log-likelihood for classification

Why does knowing the wrong probability help us measure information loss?

log-loss / cross-entropy loss penalizes: confident wrong predictions more heavily

How wrong predictions hurt more than they're supposed to

Swipe through 100 ML concepts daily

Open Pocket Polymath