Why do we sometimes zoom in faster on a scene?
Attention (machine learning)
Why do we sometimes zoom in faster on a scene?
Imagine you're watching a movie and suddenly a character you love appears on screen. You want to focus on them quickly to enjoy the moment.
Think of your eyes as a camera that can zoom in on different parts of the scene. Instead of moving slowly to focus on the character, your eyes can quickly scan the entire scene and instantly zoom in on the character, like a camera with a fast zoom feature.
Example
In the movie, your eyes quickly scan the entire scene (a 1920x1080 image) and zoom in on the character (a 32x32 pixel region) in less than a second.
Remember this
This is why flash attention is faster—it quickly scans the entire scene and zooms in on important parts, avoiding the need to build a large matrix to focus on every detail.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
paged attention (vLLM) improves serving throughput
Paged attention (vLLM) improves serving throughput by reducing latency through non-contiguous KV-cache pages, enabling faster data retrieval
Flashbulb memory
Flashbulb memories are vivid but not always accurate
Masking (behavior)
Can you not see what's right in front of you?
Her Alibi
Bruce Beresford directed Her Alibi
Matrix multiplication algorithm
Ever wondered how computers speed up multiplying huge numbers?
GAT (Graph Attention Network) adds: learned attention weights between neighbors
GATs use learned attention weights between neighbors
Swipe through 100 ML concepts daily
Open Pocket Polymath