Dropout randomly sets neuron inputs/outputs to zero during training
Dropout (neural networks)
Dropout randomly sets neuron inputs/outputs to zero during training
Dropout is a regularization technique used to prevent overfitting in neural networks by randomly disabling neurons during training. This randomness helps the network learn more robust features that are not reliant on specific neurons.
Example
During training, if a neuron has a 50% chance of being dropped out, the input to that neuron will be set to zero for that training instance.
Remember this
Dropout reduces the risk of overfitting by ensuring that the neural network does not become overly reliant on any single neuron, promoting better generalization.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
dropout works as regularization: it approximates an ensemble of subnetworks
Why does turning off neurons randomly help a brainy computer learn better?
Convolutional neural network
Can a neural network learn too well?
AdaGrad's learning rate decays to zero
Why does a car's speed drop when it goes uphill?
to standardize: when you need zero mean and unit variance for gradient-based optimization
Why do we need to make data uniform before training a model?
ill-conditioned matrices cause numerical instability: small input changes → large output changes
Ill-conditioned matrices lead to numerical instability
Machine learning
Can we teach computers to learn like humans?
Swipe through 100 ML concepts daily
Open Pocket Polymath