the condition number κ(A) measures: sensitivity of Ax=b to perturbations

How small changes affect big outcomes

Imagine you're baking a cake and the recipe calls for 1 cup of sugar. What happens if you accidentally use 1.01 cups instead?

The cake might taste slightly different, showing how sensitive the recipe is to small changes in ingredients. This concept is called the condition number.

Example

If the original recipe (1 cup) was perfect, using 1.01 cups (a 1% increase) might not ruin the cake but could subtly change its taste.

Remember this

The condition number tells you how much a small change in input can affect the final result.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

ill-conditioned matrices cause numerical instability: small input changes → large output changes

Small changes can lead to huge errors in math

the back-door criterion identifies: sufficient adjustment sets for causal estimation

Can we trust studies without random experiments?

log-loss / cross-entropy loss penalizes: confident wrong predictions more heavily

How wrong predictions hurt more than they're supposed to

Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits

How do we measure uncertainty in everyday decisions?

cross-entropy equals negative log-likelihood for classification

Why does knowing the wrong probability help us measure information loss?

temperature T in softmax(x/T) controls entropy: T→0 is argmax, T→∞ is uniform

How does adjusting T affect the certainty of choices?

Swipe through 100 ML concepts daily

Open Pocket Polymath