to write a vector addition kernel in Triton: load blocks, add, store

tl.load and tl.store do in Triton: read/write tensors from/to GPU global memory

`tl.load` reads tensors from GPU memory; `tl.store` writes tensors to GPU memory

to write a fused softmax kernel in Triton: load row, compute max, subtract, exp, sum, divide

What if we could turn raw scores into probabilities?

tl.dot does in Triton: block-level matrix multiply using tensor cores

tl.dot performs block-level matrix multiplication using tensor cores in Triton

a Triton @triton.jit decorator does: compiles a Python function into a GPU kernel

@triton.jit decorator compiles Python function into a GPU kernel

a Triton kernel is

Can your phone run faster with a different brain?

Triton differs from CUDA

Why does a super-fast computer sometimes run slower than a regular one?