OLS minimizes squared differences
Ordinary least squares
OLS minimizes squared differences
The Gauss–Markov theorem states that OLS estimators are the best linear unbiased estimators (BLUE) when certain assumptions hold true. This theorem guarantees the efficiency of OLS estimators in linear regression models.
Example
Consider a dataset with observed values (y) and predicted values (ŷ) from a linear regression model. The OLS method calculates the best-fitting line by minimizing the squared differences (y - ŷ)².
Remember this
Understanding OLS is crucial for accurate parameter estimation and model fitting in linear regression analysis.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Regularization (mathematics)
L1 regularization results in sparse solutions
t-SNE preserves local structure
Can we see the hidden patterns in a cloud of data points?
Principal component analysis
Eigenvectors point along maximum variance
ill-conditioned matrices cause numerical instability: small input changes → large output changes
Ill-conditioned matrices lead to numerical instability
Eigenvalues and eigenvectors
Eigenvectors are unchanged in direction by a linear transformation
Computational complexity of matrix multiplication
O(n³) naive matrix multiplication
Swipe through 100 ML concepts daily
Open Pocket Polymath