Eigenvectors point along maximum variance
Principal component analysis
Eigenvectors point along maximum variance
Eigenvectors in PCA represent the directions of maximum variance in the data, revealing the principal axes.
PCA transforms data into a new coordinate system where principal components capture the largest variation. These components are orthogonal unit vectors that best fit the data while minimizing squared perpendicular distances.
The principal components form an orthonormal basis, making individual dimensions uncorrelated. This basis simplifies data visualization and analysis by focusing on the most significant variation directions.
Example
In a dataset of student grades, PCA might reveal that the first principal component captures the variance due to overall academic performance, while the second captures variance related to participation in extracurricular activities.
Remember this
Understanding principal axes through eigendecomposition helps in reducing dimensionality and identifying key patterns in data.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Eigenvalues and eigenvectors
Eigenvectors are unchanged in direction by a linear transformation
PCA vs t-SNE: PCA preserves global variance linearly, t-SNE preserves local structure nonlinearly
How can we teach computers to understand what we like?
Ordinary least squares
OLS minimizes squared differences
the dot product measures alignment: it equals |a||b|cos(θ)
Why do vectors sometimes "agree" with each other?
sinusoidal position encoding works: each dimension has a different frequency
Sinusoidal position encoding assigns unique frequencies to each dimension, enabling the model to distinguish positions effectively
Dimensionality reduction
Dimensionality reduction transforms high-dimensional data into low-dimensional space while preserving meaningful properties
Swipe through 100 ML concepts daily
Open Pocket Polymath