fused kernels do

Why wait for a computer to finish one task before starting another?

Imagine you're cooking dinner. You need to chop vegetables, boil pasta, and toast bread. Doing each task separately takes a lot of time and effort.

Think of a kitchen gadget that could chop, boil, and toast all at once. This idea is called combining multiple tasks into one step.

Example

Instead of chopping vegetables (5 minutes), boiling pasta (10 minutes), and toasting bread (3 minutes) separately, a multi-functional gadget could do all three tasks simultaneously in 15 minutes.

Remember this

Combining tasks into one step saves time and effort.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

operator fusion does at the compiler level: merges adjacent ops to reduce memory traffic

Compiler optimizations merge adjacent operations to reduce memory traffic

kernel fusion reduces memory bandwidth bottleneck

Can you imagine faster video games without waiting for loading screens?

most transformer operations are memory-bound, not compute-bound

Why do computers sometimes get tired?

Loop nest optimization

Can speeding up your computer make tasks quicker?

gradient checkpointing trades: recomputes activations to save memory

Gradient checkpointing trades off computation time for memory savings by recomputing activations

Triton differs from CUDA

Why does a super-fast computer sometimes run slower than a regular one?

Swipe through 100 ML concepts daily

Open Pocket Polymath