Ever wondered how your favorite streaming service instantly starts playing a movie?
Distributed hash table
Ever wondered how your favorite streaming service instantly starts playing a movie?
Imagine you're trying to watch a movie on a streaming service, but it takes forever to load because the data is stored far away.
A distributed caching system acts like a network of mini-stores near your home, keeping copies of the movie data so it loads faster when you want to watch it.
Example
Instead of waiting 30 seconds for the movie to load (like a distant store), you get it in 5 seconds because the data is stored closer (like a mini-store near you).
Remember this
Distributed caching reduces latency by storing data closer to where it's needed.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
kernel fusion reduces memory bandwidth bottleneck
Can you imagine faster video games without waiting for loading screens?
GQA reduces KV-cache memory by the group factor
Ever wondered how websites stay fresh in search results?
LSM trees optimize: write-heavy workloads by buffering writes in memory
Ever wondered how your favorite social media app handles millions of new posts every minute?
paged attention (vLLM) improves serving throughput
Paged attention (vLLM) improves serving throughput by reducing latency through non-contiguous KV-cache pages, enabling faster data retrieval
Load balancing (computing)
Load balancing distributes tasks efficiently across resources
a CDN does: caches content at edge locations close to users
Ever wondered why YouTube videos load instantly even in remote areas?
Swipe through 100 ML concepts daily
Open Pocket Polymath