Can a tree decide the best path to take?
Decision tree
Can a tree decide the best path to take?
Imagine you're planning a picnic and trying to decide whether to go to the park or the beach. You want to choose the location with the best weather forecast for your chosen day.
A decision tree helps you make this choice by splitting your decision into smaller parts, like checking the weather forecast for each day until you find the best day for your picnic. The "best" day is determined by how much uncertainty (or randomness) there is in the weather forecast for each day.
Example
You check the weather forecast for each day of the week. On Monday, there's a 70% chance of rain. On Tuesday, there's a 30% chance of rain. The decision tree helps you choose Tuesday because it has less uncertainty (lower entropy) in the weather forecast.
Remember this
The decision tree uses entropy to split nodes in the tree, aiming to reduce uncertainty at each step.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
cross-entropy equals negative log-likelihood for classification
Why does knowing the wrong probability help us measure information loss?
O(n log n) is the lower bound for comparison-based sorting
Ever wonder why sorting can't be faster than a certain point?
UMAP is faster than t-SNE
UMAP is faster due to approximate nearest neighbors and cross-entropy optimization
t-SNE preserves local structure
Can we see the hidden patterns in a cloud of data points?
Entropy H = -Σ p(x) log₂ p(x) measures average surprise in bits
How do we measure uncertainty in everyday decisions?
Likelihood function
Why can't we just guess probabilities correctly all the time?
Swipe through more Machine Learning concepts
Open Pocket Polymath