How can we efficiently share resources without constant reorganization?
Image: EdoardoRamalli, CC BY-SA 4.0, via Wikimedia Commons
How can we efficiently share resources without constant reorganization?
Imagine you're organizing a group of friends to share a limited number of books. Each friend wants to borrow books without causing chaos when new friends join or when some leave.
Think of each book as a slot in a library. Consistent hashing helps us assign books to friends in such a way that when new friends come or go, only a few need to swap books instead of everyone.
Example
If there are 10 books (slots) and 5 friends (keys), consistent hashing means only 2 friends (10/5) need to change their book assignments when a new friend joins or an old one leaves.
Remember this
Consistent hashing minimizes the number of friends who need to swap books when the group changes, making sharing resources smoother and less disruptive.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
consistent hashing solves: minimizes key redistribution when servers are added/removed
Ever wonder why adding or removing servers doesn't mess up your favorite streaming service?
Peer-to-peer
How do you find a lost friend in a huge city?
Locality-sensitive hashing
Locality-sensitive hashing (LSH) hashes similar items into the same buckets
BPE tokenization does: iteratively merges the most frequent adjacent byte pairs
How do we make computers understand language better?
ACID
ACID guarantees data validity in transactions
BPE tokenization does: iteratively merges the most frequent byte pairs
How can we shrink text files without losing meaning?
Swipe through 100 ML concepts daily
Open Pocket Polymath