Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

You want to know if you'll need an umbrella tomorrow, but you can't wait until tomorrow to find out.

Imagine you're guessing tomorrow's weather by looking at today's sky. You make a guess, then wait for a bit to see if your guess was right. If it was wrong, you adjust your guess based on what you just learned. This way, you get better at guessing without waiting for the whole week to pass.

Example

Today is partly cloudy, so you guess there's a 50% chance of rain tomorrow. After a few hours, it starts to rain. You now know your guess was off, so you adjust your future guesses to be more accurate.

Remember this

You improve your guesses by learning from new information as it comes in, not just waiting for the end result.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

the do-calculus does: computes interventional probabilities from observational data

Can we predict the future from the past?

Partially observable Markov decision process

How can you make the best decisions when you can't see everything?

Markov decision process

Ever wondered how a robot decides its next move in a maze?

Reinforcement learning from human feedback

RLHF optimizes a reward model trained on human preference pairs

Adam optimizer weight update with m and v terms

Ever wondered how we can predict complex financial markets with high accuracy?

Adam combines momentum and RMSprop: adapts per-parameter learning rates

Ever wondered how a car adjusts its speed for smooth turns?

Swipe through 100 ML concepts daily

Open Pocket Polymath