Context window limit refers to the maximum number of tokens a model can process at once
Image: Santa Clara Valley Transportation Authority, Public domain, via Wikimedia Commons
Glossary of poker terms
Context window limit refers to the maximum number of tokens a model can process at once
The context window limit is a crucial aspect of language models, determining how much input they can handle simultaneously. This limit affects the model's ability to understand and generate coherent responses.
Example
If a language model has a context window limit of 512 tokens, it can process up to 512 words or phrases at once, depending on the average length of the tokens.
Remember this
Understanding the context window limit is essential for optimizing the performance of language models in various applications.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
register pressure means: too many variables per thread reduces occupancy
Too many variables per thread reduces occupancy
the compute-optimal training ratio is: roughly 20 tokens per parameter
Compute-optimal training ratio: roughly 20 tokens per parameter
the vocabulary size matters: larger vocab = shorter sequences but more parameters
Larger vocab leads to shorter sequences but more parameters
Neural scaling law
Chinchilla scaling law: optimal model size scales linearly with compute budget
Attention Is All You Need
"Attention Is All You Need" introduced the transformer architecture in 2017
BLOCK_SIZE means in Triton: the tile size each program instance processes
Block size refers to the minimal unit of data for block ciphers
Swipe through 100 ML concepts daily
Open Pocket Polymath