Why does a super-fast computer sometimes run slower than a regular one?
Image: CC BY-SA 2.5, via Wikimedia Commons
Triton differs from CUDA
Why does a super-fast computer sometimes run slower than a regular one?
Imagine you're waiting for a friend who lives 10 miles away. If you walk, you get there in 1 hour. But if you teleport, you instantly arrive. Now, imagine a computer doing calculations. If it uses one method, it might take a long time. If it uses another method, it can finish much faster.
Computers can do things in different ways. Sometimes, they use a method that lets them work on lots of small tasks at once, like a group of friends working together on a puzzle. This is called parallel computing. The other method makes them work on one big task at a time, like doing a puzzle alone. This is called serial computing.
Example
If you have 10 friends helping you solve a puzzle, it takes 10 minutes. If you do it alone, it takes 100 minutes. In computers, using lots of small tasks (parallel computing) can finish a job much faster than doing one big task (serial computing).
Remember this
Parallel computing lets computers finish tasks faster by dividing them into smaller parts and working on them at the same time.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
a Triton kernel is
Can your phone run faster with a different brain?
Loop nest optimization
Can speeding up your computer make tasks quicker?
Triton auto-tunes BLOCK_SIZE: different sizes optimize for different hardware
Rate-distortion optimization balances video quality and file size
CUDA
CUDA enables parallel computation on GPUs
tensor cores are
Why can computers crunch numbers faster than humans?
most transformer operations are memory-bound, not compute-bound
Why do computers sometimes get tired?
Swipe through 100 ML concepts daily
Open Pocket Polymath