the trace equals the sum of eigenvalues: tr(A) = Σλ_i

How can a vector stay the same after a transformation?

Imagine you're stretching a rubber band. You pull it from both ends, but it only stretches along one direction. Why doesn't it stretch in all directions?

Think of a rubber band as a vector. When you pull it, you're applying a transformation. Some vectors, called eigenvectors, only stretch or shrink by a certain amount, not change direction. This amount is called an eigenvalue.

Example

If you pull the rubber band in a certain direction, it stretches by a factor of 2. So, if the original length was 1 unit, now it's 2 units. That's like saying the eigenvalue λ is 2.

Remember this

The trace of a matrix (tr(A)) equals the sum of its eigenvalues because it's like adding up how much each vector stretches or shrinks when transformed.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

the dot product measures alignment: it equals |a||b|cos(θ)

Why do vectors sometimes "agree" with each other?

Eigenvalues and eigenvectors

Eigenvectors are unchanged in direction by a linear transformation

Principal component analysis

Eigenvectors point along maximum variance

KL divergence is always ≥ 0 and equals 0 only when P = Q exactly

Why can't we just compare two things directly?

Matrix norm

L1 norm of a vector is the sum of absolute values of its components

the momentum term does: v_t = βv_{t-1} + ∇L, accumulates gradient direction

Momentum term accelerates convergence in the gradient direction

Swipe through 100 ML concepts daily

Open Pocket Polymath