Why can't computers talk to each other as fast as we want?
History of general-purpose CPUs
Why can't computers talk to each other as fast as we want?
Imagine you're trying to send a photo to a friend on a slow internet connection. You hit "send," but it takes forever for the photo to arrive.
Computers have a slow way of sending information between the brain (CPU) and the memory (RAM). It's like trying to pour water through a narrow straw.
Example
If sending 1 MB of data takes 1 second through the straw, sending 10 MB would take 10 seconds.
Remember this
The Von Neumann bottleneck slows down data transfer, causing delays in processing tasks.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Von Neumann architecture
CPU must fetch both data and instructions from memory
kernel fusion reduces memory bandwidth bottleneck
Can you imagine faster video games without waiting for loading screens?
Loop nest optimization
Can speeding up your computer make tasks quicker?
warp divergence kills performance
Warp divergence causes threads to execute non-uniformly, leading to idle cycles and reduced throughput
most transformer operations are memory-bound, not compute-bound
Why do computers sometimes get tired?
Triton differs from CUDA
Why does a super-fast computer sometimes run slower than a regular one?
Swipe through 100 ML concepts daily
Open Pocket Polymath