Markov decision process

Ever wondered how a robot decides its next move in a maze?

Imagine a robot navigating a maze to find an exit. It can't see the whole maze at once and must make decisions based on limited information.

The robot uses a rule to choose its next step, considering the immediate outcomes of its actions. This rule helps it make the best choice at each point to reach the exit.

Example

If the robot is at a junction with one path leading to a dead end and another to a room with two more exits, it chooses the path with two exits, hoping it leads closer to the exit.

Remember this

The robot's decision-making process is guided by the Bellman equation, which helps it evaluate the best action to take at any given moment.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Partially observable Markov decision process

How can you make the best decisions when you can't see everything?

Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

Hidden Markov model

Ever wondered how you can predict outcomes without seeing all the details?

Travelling salesman problem

Can we always find the shortest path visiting all cities?

Markov chain

Why does flipping a coin repeatedly get predictable?

Causality

Can time-traveling cause a future event?

Swipe through more Machine Learning concepts

Open Pocket Polymath