Why do random points in high dimensions seem to be evenly spaced?
Image: Ann 2000, CC BY-SA 4.0, via Wikimedia Commons
Why do random points in high dimensions seem to be evenly spaced?
Imagine throwing darts at a wall with many holes; no matter where you stand, you expect them to hit the holes evenly.
As you throw more darts into a high-dimensional space, they tend to spread out evenly, covering the space uniformly.
Example
If you throw 100 darts at a wall with 10 holes, you'll likely hit each hole around 10 times.
Remember this
Random points in high dimensions are nearly equidistant because they evenly cover the space.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
random projection to O(log n/ε²) dimensions preserves pairwise distances within 1±ε
Can you shrink a 3D object into 2D without losing its shape?
the Johnson-Lindenstrauss lemma says
Can we shrink big data without losing important details?
cosine similarity works better than Euclidean distance in high dimensions
Cosine similarity measures orientation, not magnitude, making it more robust to irrelevant dimensions in high-dimensional spaces
saddle points are more common than local minima in high dimensions
Why do some mountains have flat tops instead of peaks or valleys?
the dot product measures alignment: it equals |a||b|cos(θ)
Why do vectors sometimes "agree" with each other?
Manifold hypothesis
High-dimensional data lies on lower-dimensional manifolds
Swipe through 100 ML concepts daily
Open Pocket Polymath