Definite matrix

How do we measure the steepness of a hill in all directions?

Imagine you're hiking and want to know how steep the hill is in every direction from your current position.

Think of the hill as a 3D graph. The steepness in every direction tells us how quickly the elevation changes. This idea is captured by a special matrix called the Hessian matrix.

Example

If you're at point (x, y) on the hill, the Hessian matrix tells you how steep the hill is in the x and y directions, as well as how these directions affect each other.

Remember this

The Hessian matrix helps us understand the curvature of the hill at any point, revealing how steep it is in all directions.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Hessian matrix

The Hessian matrix is denoted by H or ∇²

natural gradient descent does: preconditions with inverse Fisher matrix

Ever wondered how to climb a steep mountain without measuring every step?

Cholesky decomposition

Cholesky decomposition factors A = LL^T for symmetric positive definite matrices

Lagrangian L(x,λ) = f(x) - λg(x)

How do we find the path an object naturally takes?

Gradient

Gradient points uphill in the direction of steepest increase of f

Eigenvalues and eigenvectors

Eigenvectors are unchanged in direction by a linear transformation

Swipe through 100 ML concepts daily

Open Pocket Polymath