soft targets carry more information than hard labels: they encode class similarities

Why do some learning methods need to explore more than others?

Imagine you're trying to find the best coffee shop in town. You know some shops are good (labeled data), but you want to discover hidden gems (unlabeled data) that might be even better.

Reinforcement learning (RL) is like exploring different coffee shops to find the best one. Unlike supervised learning (which uses known good shops) or unsupervised learning (which finds patterns in all shops), RL experiments with new choices (exploration) and uses what it knows (exploitation) to find the top spot.

Example

You visit a new coffee shop (exploration) and try different drinks (actions) to see if it's better than the ones you already know about (exploitation). Over time, you find the perfect blend (optimal action) that maximizes your coffee pleasure (reward).

Remember this

Soft targets in RL carry more information because they help the agent learn about new possibilities (exploration) and improve its choices (exploitation).

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Knowledge distillation

Knowledge distillation transfers knowledge from a large model to a smaller one without loss of validity

Prompt engineering

The GenAI model learns tasks from examples in the prompt

instruction tuning does: fine-tunes on (instruction, response) pairs

Fine-tuning adapts pre-trained models to new tasks

Retrieval-augmented generation

RAG enables LLMs to access new information without retraining

mean pooling often outperforms [CLS] for sentence similarity tasks

Mean pooling captures overall sentence meaning better than [CLS] token embedding

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

Swipe through 100 ML concepts daily

Open Pocket Polymath