Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

You want to know if you'll need an umbrella tomorrow, but you can't wait until tomorrow to find out.

Imagine you're guessing tomorrow's weather by looking at today's sky. You make a guess, then wait for a bit to see if your guess was right. If it was wrong, you adjust your guess based on what you just learned. This way, you get better at guessing without waiting for the whole week to pass.

Example

Today is partly cloudy, so you guess there's a 50% chance of rain tomorrow. After a few hours, it starts to rain. You now know your guess was off, so you adjust your future guesses to be more accurate.

Remember this

You improve your guesses by learning from new information as it comes in, not just waiting for the end result.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

instruction tuning does: fine-tunes on (instruction, response) pairs

Fine-tuning adapts pre-trained models to new tasks

Reinforcement learning from human feedback

RLHF optimizes a reward model trained on human preference pairs

Adam optimizer weight update with m and v terms

Ever wondered how we can predict complex financial markets with high accuracy?

Adam combines momentum and RMSprop: adapts per-parameter learning rates

Ever wondered how a car adjusts its speed for smooth turns?

Proximal gradient methods for learning

Why can't we always find the best path in a maze?

Prompt engineering

The GenAI model learns tasks from examples in the prompt

Swipe through 100 ML concepts daily

Open Pocket Polymath