ill-conditioned matrices cause numerical instability: small input changes → large output changes

Small changes can lead to huge errors in math

Imagine you're baking a cake and you accidentally add a tiny bit too much flour. The cake still tastes good, but it's not perfect.

In math, if you start with a nearly perfect recipe (matrix), adding a tiny mistake (small input change) can make the final result (output) much worse (large output change).

Example

You add 0.01 cups extra flour to a 10-cup recipe. The cake's texture changes dramatically.

Remember this

Small input changes in ill-conditioned matrices can lead to large output changes, causing numerical instability.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

the condition number κ(A) measures: sensitivity of Ax=b to perturbations

How small changes affect big outcomes

Neural network (machine learning)

Ever tried adjusting the learning rate like tuning a musical instrument?

Adam has bias correction: divides by (1-β^t) in early steps

Why do we sometimes need to fix mistakes in computer decisions?

Overlapping subproblems

Ever calculated a huge Fibonacci sequence by hand?

Strength reduction

Why do computers sometimes get tired of doing the same math over and over?

Linear multistep method

Why does skipping steps save time but lose accuracy?

Swipe through 100 ML concepts daily

Open Pocket Polymath