the L1 norm is not differentiable at zero

Absolute value's kink makes it non-differentiable at zero

Imagine you're measuring the distance from your home to a friend's house. You want to know how fast you're getting closer as you walk.

The distance changes smoothly until you reach your friend's house, but there's a sudden stop at the exact point of arrival. This stop is like the kink in the absolute value function at zero.

Example

As you walk towards your friend, the distance decreases smoothly until you reach them. At that exact moment, there's an abrupt halt in the distance change.

Remember this

The absolute value function's kink at zero makes it non-differentiable there.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Norm (mathematics)

L∞ norm equals max absolute value

the Lp norm ball shape changes as p goes from 1 to 2 to infinity

How does the shape of a ball change as we measure distance differently?

Matrix norm

L1 norm of a vector is the sum of absolute values of its components

KL divergence is always ≥ 0 and equals 0 only when P = Q exactly

Why can't we just compare two things directly?

Kullback–Leibler divergence

KL divergence is not symmetric: D_KL(P||Q) ≠ D_KL(Q||P)

Dynamical system

A fixed point is where dx/dt = 0

Swipe through 100 ML concepts daily

Open Pocket Polymath