instruction tuning does: fine-tunes on (instruction, response) pairs

Fine-tuning adapts pre-trained models to new tasks

For efficient fine-tuning, lightweight modules called "adapters" can be inserted into the model's architecture. These adapters nudge the embedding space for domain adaptation and can be fine-tuned by tuning only their weights, leaving the rest of the model's weights frozen.

Example

A convolutional neural network (CNN) used for image classification can be fine-tuned for a specific task like medical image diagnosis by keeping the earlier layers frozen and only fine-tuning the later layers.

Remember this

Fine-tuning allows for efficient adaptation of pre-trained models to new tasks without retraining the entire network.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Prompt engineering

The GenAI model learns tasks from examples in the prompt

gradient accumulation simulates larger batch sizes without more memory

Can you train a machine like you do with a computer?

soft targets carry more information than hard labels: they encode class similarities

Why do some learning methods need to explore more than others?

gradient checkpointing trades: recomputes activations to save memory

Gradient checkpointing trades off computation time for memory savings by recomputing activations

Machine learning

Can we teach computers to learn like humans?

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

Swipe through 100 ML concepts daily

Open Pocket Polymath