
How do we choose the best model for our data?
Image: GruenerBogen, CC BY-SA 4.0, via Wikimedia Commons
How do we choose the best model for our data?
Imagine you're trying to predict the weather. You have many different models that use various data points like temperature, humidity, and wind speed. You want to pick the model that gives you the most accurate predictions without being too complicated.
Think of each model as a different recipe for predicting the weather. Some recipes might be very detailed (complex) and work well for today's weather but not for tomorrow's. Others might be simpler and work okay for a variety of conditions. We want to find the recipe that gives us good predictions without being too complicated or too simple.
Example
Let's say Model A uses 10 variables and Model B uses 5. Model A fits today's weather perfectly but fails tomorrow. Model B fits moderately well for both days but is simpler.
Remember this
The Fisher Information Matrix helps us balance the trade-off between a model's accuracy (goodness of fit) and its simplicity (simplicity), guiding us to the best recipe for predicting the weather.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Fisher information
Fisher information measures information about unknown parameters
Maximum likelihood estimation
How do we find the best-fitting parameters for a model?
Lasso (statistics)
Ever wondered how to pick the best features for a machine learning model?
Softmax function
Softmax converts real numbers into a probability distribution
Bayes' theorem
Bayes' theorem formula: P(A|B) = [P(B|A) * P(A)] / P(B)
Variational autoencoder
ELBO formula in variational inference
Swipe through more Machine Learning concepts
Open Pocket Polymath