instruction tuning does: fine-tunes on (instruction, response) pairs

Ever wondered how a chef can master a new cuisine quickly?

Imagine a chef who's an expert in Italian cooking wants to start making French dishes. They already know a lot about cooking but need to adapt to new recipes and techniques.

Fine-tuning is like giving the chef a few new recipes to practice, while keeping the rest of their cooking skills intact. They tweak just the new parts without messing up what they already know.

Example

The chef already knows how to make pasta (pre-trained knowledge), but now they fine-tune their skills to make croissants (downstream task) by practicing only the new recipe (fine-tuning weights).

Remember this

Fine-tuning allows experts to quickly adapt to new tasks by adjusting only the relevant parts of their knowledge.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Neural network (machine learning)

Ever tried adjusting the learning rate like tuning a musical instrument?

Adam vs SGD: Adam adapts per-parameter rates, SGD often generalizes better with tuning

Adam adjusts learning rates per-parameter, SGD generalizes better with tuning

instruction-level parallelism (ILP) achieves: multiple operations per clock cycle

Ever wondered how computers can do so many tasks at once?

gradient accumulation simulates larger batch sizes without more memory

Can you train a machine like you do with a computer?

Loop nest optimization

Can speeding up your computer make tasks quicker?

Reasoning model

RLMs excel in logic, math, and programming tasks

Swipe through 100 ML concepts daily

Open Pocket Polymath