parallel transport does: moves vectors along a curve while preserving their properties

How can we keep a vector's direction constant while moving along a path?

Imagine walking along a winding path in a park, holding a straight stick upright. You want to keep the stick's orientation the same as you move.

Parallel transport is like walking with a straight stick, keeping it upright as you move along a curved path. It's a way to move vectors (like the stick) along a curve without changing their direction relative to the curve.

Example

As you walk along the path, you rotate the stick to stay upright, but the stick's orientation remains constant relative to the path.

Remember this

Parallel transport keeps vectors' directions constant along a curve, preserving their properties.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

the trace equals the sum of eigenvalues: tr(A) = Σλ_i

How can a vector stay the same after a transformation?

Vanishing gradient problem

Residual connections help by allowing gradient flow through the skip connection

the momentum term does: v_t = βv_{t-1} + ∇L, accumulates gradient direction

Momentum term accelerates convergence in the gradient direction

Pre-LN transformers are easier to train

Pre-LN transformers use residual connections, allowing gradients to flow more smoothly during backpropagation

the dot product measures alignment: it equals |a||b|cos(θ)

Why do vectors sometimes "agree" with each other?

non-convex loss landscapes are hard: many local minima and saddle points

Why do some hills have more tricky paths than others?

Swipe through 100 ML concepts daily

Open Pocket Polymath