soft targets carry more information than hard labels: they encode class similarities

Why do some learning methods need to explore more than others?

Imagine you're trying to find the best coffee shop in town. You know some shops are good (labeled data), but you want to discover hidden gems (unlabeled data) that might be even better.

Reinforcement learning (RL) is like exploring different coffee shops to find the best one. Unlike supervised learning (which uses known good shops) or unsupervised learning (which finds patterns in all shops), RL experiments with new choices (exploration) and uses what it knows (exploitation) to find the top spot.

Example

You visit a new coffee shop (exploration) and try different drinks (actions) to see if it's better than the ones you already know about (exploitation). Over time, you find the perfect blend (optimal action) that maximizes your coffee pleasure (reward).

Remember this

Soft targets in RL carry more information because they help the agent learn about new possibilities (exploration) and improve its choices (exploitation).

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Knowledge distillation

Knowledge distillation transfers knowledge from a large model to a smaller one without loss of validity

Prompt engineering

The GenAI model learns tasks from examples in the prompt

Retrieval-augmented generation

RAG enables LLMs to access new information without retraining

2022 in science

Why do Transformers sometimes seem to 'ignore' irrelevant parts of the input?

mean pooling often outperforms [CLS] for sentence similarity tasks

Mean pooling captures overall sentence meaning better than [CLS] token embedding

score matching does: learns the gradient of the log-density without normalizing

Ever wonder how we can compare apples and oranges fairly in studies?

Swipe through 100 ML concepts daily

Open Pocket Polymath