Partially observable Markov decision process

How can you make the best decisions when you can't see everything?

Imagine you're driving in foggy weather and can't see the road clearly. You need to decide when to slow down to avoid an accident.

You can't see the road, so you rely on your past experiences and the limited visibility to make safer decisions. This is like using a formula to guide you when you don't have all the information.

Example

If you've driven this way before and it was safe, you might decide to continue driving slowly until the fog clears.

Remember this

The Bellman equation helps you calculate the best action to take based on what you know and what you've experienced, even when you can't see everything.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits

How do we measure uncertainty in everyday decisions?

Markov chain Monte Carlo

MCMC samples from complex posterior distributions

Metropolis–Hastings algorithm

Metropolis-Hastings algorithm samples from difficult distributions

the Dirichlet distribution does: distribution over probability simplices

How do we predict the likelihood of various outcomes in uncertain situations?

Fisher information

Fisher information measures information about unknown parameters

Swipe through 100 ML concepts daily

Open Pocket Polymath