How can you add numbers safely in a busy computer?
List of gay characters in animation
How can you add numbers safely in a busy computer?
Imagine you're at a busy coffee shop, and you need to add up everyone's orders to know how much money is collected. But there's a catch: you can't just grab a pen and start scribbling numbers because someone else might be adding their orders at the same time.
Think of a shared notebook where everyone writes their orders. You need a way to make sure everyone's additions don't mess up the total. AtomicAdd is like a magic pen that writes down numbers without letting others interfere.
Example
If Alice adds 5, Bob adds 3, and Charlie tries to add $2 at the same time, AtomicAdd ensures the total still comes out right.
Remember this
AtomicAdd ensures accurate and safe addition of numbers in shared memory.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
__syncthreads() does in CUDA: synchronizes all threads within a block
Why do computers sometimes work faster when you click "OK"?
Thread block (CUDA programming)
Thread blocks can contain up to 1024 threads as of March 2010
CUDA
CUDA enables parallel computation on GPUs
Dynamic random-access memory
DRAM requires periodic refreshing to maintain data integrity
cooperative groups enable in CUDA: flexible thread synchronization patterns
Cooperative groups enable flexible thread synchronization patterns in CUDA
Triton differs from CUDA
Why does a super-fast computer sometimes run slower than a regular one?
Swipe through 100 ML concepts daily
Open Pocket Polymath