Partially observable Markov decision process

How can you make the best decisions when you can't see everything?

Imagine you're driving in foggy weather and can't see the road clearly. You need to decide when to slow down to avoid an accident.

You can't see the road, so you rely on your past experiences and the limited visibility to make safer decisions. This is like using a formula to guide you when you don't have all the information.

Example

If you've driven this way before and it was safe, you might decide to continue driving slowly until the fog clears.

Remember this

The Bellman equation helps you calculate the best action to take based on what you know and what you've experienced, even when you can't see everything.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Markov decision process

Ever wondered how a robot decides its next move in a maze?

Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits

How do we measure uncertainty in everyday decisions?

Markov chain Monte Carlo

MCMC samples from complex posterior distributions

Hidden Markov model

Ever wondered how you can predict outcomes without seeing all the details?

Metropolis–Hastings algorithm

Metropolis-Hastings algorithm samples from difficult distributions

Swipe through 100 ML concepts daily

Open Pocket Polymath