Triton differs from CUDA

Why does a super-fast computer sometimes run slower than a regular one?

Imagine you're waiting for a friend who lives 10 miles away. If you walk, you get there in 1 hour. But if you teleport, you instantly arrive. Now, imagine a computer doing calculations. If it uses one method, it might take a long time. If it uses another method, it can finish much faster.

Computers can do things in different ways. Sometimes, they use a method that lets them work on lots of small tasks at once, like a group of friends working together on a puzzle. This is called parallel computing. The other method makes them work on one big task at a time, like doing a puzzle alone. This is called serial computing.

Example

If you have 10 friends helping you solve a puzzle, it takes 10 minutes. If you do it alone, it takes 100 minutes. In computers, using lots of small tasks (parallel computing) can finish a job much faster than doing one big task (serial computing).

Remember this

Parallel computing lets computers finish tasks faster by dividing them into smaller parts and working on them at the same time.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

a Triton kernel is

Can your phone run faster with a different brain?

instruction-level parallelism (ILP) achieves: multiple operations per clock cycle

Ever wondered how computers can do so many tasks at once?

Loop nest optimization

Can speeding up your computer make tasks quicker?

CUDA

CUDA enables parallel computation on GPUs

BLOCK_SIZE means in Triton: the tile size each program instance processes

Why does your computer sometimes feel slow when you're playing games?

tensor cores are

Why can computers crunch numbers faster than humans?

Swipe through 100 ML concepts daily

Open Pocket Polymath