Intrinsic dimension

Intrinsic dimension M satisfies 0 ≤ M ≤ N

The intrinsic dimension of a dataset is a measure of its complexity, indicating the minimal number of variables needed to represent it. This concept helps in understanding the underlying structure of data and signals.

Example

A dataset with 100 variables might have an intrinsic dimension of 10, meaning it can be effectively represented with just 10 variables.

Remember this

Understanding intrinsic dimension helps in efficient data compression and analysis, reducing computational costs and improving clarity.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Manifold hypothesis

High-dimensional data lies on lower-dimensional manifolds

the Johnson-Lindenstrauss lemma says

Can we shrink big data without losing important details?

Dimensionality reduction

Dimensionality reduction transforms high-dimensional data into low-dimensional space while preserving meaningful properties

cosine similarity is preferred over dot product for normalized embeddings

Why do we need a special way to measure similarity in high-dimensional spaces?

cosine similarity works better than Euclidean distance in high dimensions

Cosine similarity measures orientation, not magnitude, making it more robust to irrelevant dimensions in high-dimensional spaces

Ordinary least squares

OLS minimizes squared differences

Swipe through 100 ML concepts daily

Open Pocket Polymath