Markov decision process

Ever wondered how a robot decides its next move in a maze?

Imagine a robot navigating a maze to find an exit. It can't see the whole maze at once and must make decisions based on limited information.

The robot uses a rule to choose its next step, considering the immediate outcomes of its actions. This rule helps it make the best choice at each point to reach the exit.

Example

If the robot is at a junction with one path leading to a dead end and another to a room with two more exits, it chooses the path with two exits, hoping it leads closer to the exit.

Remember this

The robot's decision-making process is guided by the Bellman equation, which helps it evaluate the best action to take at any given moment.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Partially observable Markov decision process

How can you make the best decisions when you can't see everything?

Write the Bellman equation for reinforcement learning

Predicting tomorrow's weather with today's clues

Travelling salesman problem

Can we always find the shortest path visiting all cities?

Kolmogorov complexity

Kolmogorov complexity is uncomputable

Proximal gradient methods for learning

Why can't we always find the best path in a maze?

Physics-informed neural networks

Neural ODEs model continuous-time dynamics with a neural network as the derivative

Swipe through 100 ML concepts daily

Open Pocket Polymath