to use AUC-ROC: comparing classifiers across all thresholds

Why can't we always trust a yes-or-no answer?

Imagine you're trying to decide if a person is a good friend based on a single yes-or-no question. You might miss out on someone who could be great because you didn't ask enough questions.

A single yes-or-no question is like checking only one point on a graph to judge a friend's quality. We need to look at the whole graph, which shows how good the friend is at different levels of friendliness, to make a better decision.

Example

If you ask someone "Are you a good friend?" and they say yes, you might think they're great friends. But what if they're just good at saying yes? You need to ask more questions at different levels of friendliness to get the full picture.

Remember this

The AUC-ROC curve helps us see how well a decision rule works at all levels, not just at one point.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Receiver operating characteristic

Ever wondered how doctors decide if a test really finds cancer?

F-score

Why is one metric not enough to judge a binary classification model's performance?

to use random forests: when you want a strong baseline with minimal hyperparameter tuning

Why not always use the best single tree for predictions?

Random forest

How can we make a forest of decision trees smarter and more accurate?

score matching does: learns the gradient of the log-density without normalizing

Ever wonder how we can compare apples and oranges fairly in studies?

Binary classification

Why is it important to know if something is a disease or not?

Swipe through more Machine Learning concepts

Open Pocket Polymath