How does adjusting T affect the certainty of choices?
Image: Trondheim Havn from Trondheim, Norway, CC BY-SA 2.0, via Wikimedia Commons
temperature T in softmax(x/T) controls entropy: T→0 is argmax, T→∞ is uniform
How does adjusting T affect the certainty of choices?
Imagine you're choosing a meal from a menu, and you want to be sure about your choice.
If T is very small, you're almost sure to pick the best option, like a top chef's recommendation. But if T is very large, you're just picking randomly, like flipping a coin for each dish.
Example
With T=1, you might have a 70% chance of picking your favorite dish, but with T=100, it's like choosing blindly.
Remember this
Small T makes you confident in your choice, while large T makes it random.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits
How do we measure uncertainty in everyday decisions?
cross-entropy equals negative log-likelihood for classification
Why does knowing the wrong probability help us measure information loss?
log-loss / cross-entropy loss penalizes: confident wrong predictions more heavily
Cross-entropy loss penalizes confident wrong predictions more heavily
Entropy (information theory)
Entropy of a fair coin is 1 bit
Adam has bias correction: divides by (1-β^t) in early steps
Why do we sometimes need to fix mistakes in computer decisions?
Shannon's source coding theorem: you can't compress below entropy
Can you squeeze endless text into fewer bits without losing anything?
Swipe through 100 ML concepts daily
Open Pocket Polymath