tl.program_id(0) returns: the index of the current parallel block
Image: Svein-Magne Tunli, CC BY-SA 3.0, via Wikimedia Commons
tl.program_id(0) returns: the index of the current parallel block
tl.arange(0, BLOCK_SIZE) creates: a range of indices within the current block
Python's annual release cycle
tl.dot does in Triton: block-level matrix multiply using tensor cores
tl.dot performs block-level matrix multiplication using tensor cores in Triton
BLOCK_SIZE means in Triton: the tile size each program instance processes
Block size refers to the minimal unit of data for block ciphers
tl.load and tl.store do in Triton: read/write tensors from/to GPU global memory
`tl.load` reads tensors from GPU memory; `tl.store` writes tensors to GPU memory
Triton differs from CUDA
Why does a super-fast computer sometimes run slower than a regular one?
tl.where(mask, x, 0) does: conditional select to handle boundary conditions
`tl.where(mask, x, 0) = x if mask else 0`
Swipe through 100 ML concepts daily
Open Pocket Polymath