A fair die has entropy of log₂(6) ≈ 2.58 bits

How much information do you need to guess a die roll?

Imagine you're playing a board game with a six-sided die. You want to know how many bits of information you'd need to describe the outcome of each roll.

Think of each possible outcome as a unique story. The more stories there are, the more bits you need to tell them. For a die, there are six outcomes, so you need log₂(6) bits to describe any roll.

Example

If you had 2 outcomes (like flipping a coin), you'd need log₂(2) = 1 bit. But with 6 outcomes (like rolling a die), you need log₂(6) ≈ 2.58 bits.

Remember this

Entropy measures the average level of uncertainty or information needed to describe the outcomes of a random variable.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits

How do we measure uncertainty in everyday decisions?

Information theory

How much do you know before flipping a coin?

Entropy (information theory)

Entropy of a fair coin is 1 bit

cross-entropy equals negative log-likelihood for classification

Why does knowing the wrong probability help us measure information loss?

Rényi entropy

How do we measure the unpredictability of an event?

Entropy in thermodynamics and information theory

Ever wondered how computers decide what's important in a message?

Swipe through more Machine Learning concepts

Open Pocket Polymath