autoencoders learn the data manifold

Vector quantization

How can you store a huge library of books in a tiny closet?

weight tying does in language models: shares embedding and output projection matrices

Ever wonder how machines understand the sequence of words in a sentence?

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

the embedding layer does: maps discrete token IDs to dense learned vectors

Embeddings convert token IDs to dense vectors for neural network processing

quantization to INT8 doubles throughput

Quantization to INT8 doubles throughput because tensor cores process INT8 2x faster

cosine similarity is preferred over dot product for normalized embeddings

Why do we need a special way to measure similarity in high-dimensional spaces?