to use AUC-ROC: comparing classifiers across all thresholds

Why can't we always trust a yes-or-no answer?

Imagine you're trying to decide if a person is a good friend based on a single yes-or-no question. You might miss out on someone who could be great because you didn't ask enough questions.

A single yes-or-no question is like checking only one point on a graph to judge a friend's quality. We need to look at the whole graph, which shows how good the friend is at different levels of friendliness, to make a better decision.

Example

If you ask someone "Are you a good friend?" and they say yes, you might think they're great friends. But what if they're just good at saying yes? You need to ask more questions at different levels of friendliness to get the full picture.

Remember this

The AUC-ROC curve helps us see how well a decision rule works at all levels, not just at one point.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Receiver operating characteristic

Ever wondered how doctors decide if a test really finds cancer?

to use random forests: when you want a strong baseline with minimal hyperparameter tuning

Why not always use the best single tree for predictions?

score matching does: learns the gradient of the log-density without normalizing

Ever wonder how we can compare apples and oranges fairly in studies?

soft targets carry more information than hard labels: they encode class similarities

Why do some learning methods need to explore more than others?

Adam has bias correction: divides by (1-β^t) in early steps

Why do we sometimes need to fix mistakes in computer decisions?

Boosting (machine learning)

Boosting reduces bias in ML models

Swipe through 100 ML concepts daily

Open Pocket Polymath