QLoRA adds

Ever wondered how computers can understand and mimic human speech?

Imagine you're trying to teach a computer to recognize different voices. It's like trying to train a dog to bark when it hears your voice.

It's like giving the computer a smaller, simpler set of sounds to learn from, instead of the whole range of human speech. This way, it can focus on learning the differences between voices more efficiently. The punchline is learning vector quantization (LVQ).

Example

Instead of teaching the computer all 100,000 sounds in human speech, we give it 40 sounds to learn from, making it easier for the computer to pick up on the unique qualities of each voice.

Remember this

LVQ simplifies the learning process for computers, helping them understand and mimic human speech more effectively.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Large language model

LLMs can generate, summarize, translate, and analyze text in many contexts

AWQ does differently

AWQ selectively retains weights crucial for model performance, unlike traditional quantization

LoRA (machine learning)

LoRA uses r << d for efficient adaptation

weight tying does in language models: shares embedding and output projection matrices

Ever wonder how machines understand the sequence of words in a sentence?

Rate-distortion theory: minimum bits to represent data within distortion D

How many bits do we need to perfectly copy a song?

quantization to INT8 doubles throughput

Quantization to INT8 doubles throughput because tensor cores process INT8 2x faster

Swipe through 100 ML concepts daily

Open Pocket Polymath