Can a neural network learn too well?
Image: O'Connor P, Neil D, Liu S, Delbruck T, Pfeiffer M, CC BY 3.0, via Wikimedia Commons
Convolutional neural network
Can a neural network learn too well?
Imagine you're memorizing a new language. You start to remember every word perfectly, but suddenly, you can't understand new sentences because you're relying too much on memorized phrases.
Dropout regularization is like taking breaks while memorizing a language to avoid relying too much on memorized phrases. This helps you understand new sentences better.
Example
If you practice 100 words a day, taking a break every 10 words helps you remember them better.
Remember this
Dropout helps neural networks generalize better by preventing them from relying too much on specific features.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
dropout works as regularization: it approximates an ensemble of subnetworks
Why does turning off neurons randomly help a brainy computer learn better?
Overfitting
Why does fitting too much detail to data ruin predictions?
Deep learning
Why do smart phones sometimes struggle to recognize new faces?
Unsupervised learning
How can we find hidden patterns in massive, unlabeled data?
batch size affects generalization: larger batches find sharper minima
Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates
LASSO uses L1 to do feature selection by driving coefficients to exactly zero
Why do some numbers disappear when solving complex problems?
Swipe through more Machine Learning concepts
Open Pocket Polymath