Vector quantization

How can you store a huge library of books in a tiny closet?

Imagine you have a huge collection of books, but your closet is too small to fit them all. You need a way to store them efficiently without losing any information.

Think of your books as vectors, and your closet as storage space. Product quantization helps you store your books by dividing them into smaller groups (subvectors) and finding the most representative book (centroid) for each group. This way, you can fit more books into your closet without having to store every single one.

Example

You have 100 books and 10 shelves. Instead of placing each book on a separate shelf, you group them by genre. Each shelf now holds 10 books, but you only need to remember 10 centroids (representative books) instead of 100.

Remember this

Product quantization compresses vectors by splitting them into subvectors and quantizing each, allowing you to store more information in less space.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

GPTQ quantization does

Post-training quantization using second-order information for model compression

cosine similarity is preferred over dot product for normalized embeddings

Why do we need a special way to measure similarity in high-dimensional spaces?

autoencoders learn the data manifold

Autoencoders compress data manifold by forcing information through a bottleneck layer, learning efficient representations

the Johnson-Lindenstrauss lemma says

Can we shrink big data without losing important details?

quantization to INT8 doubles throughput

Quantization to INT8 doubles throughput because tensor cores process INT8 2x faster

weight tying does in language models: shares embedding and output projection matrices

Ever wonder how machines understand the sequence of words in a sentence?

Swipe through 100 ML concepts daily

Open Pocket Polymath