most transformer operations are memory-bound, not compute-bound

Why do computers sometimes get tired?

Imagine you're playing a video game on your computer. Suddenly, it starts to lag, and your character moves slower than before.

It's like your computer is running out of energy to think fast. It's not about doing more calculations; it's about remembering things quickly.

Example

When you try to open many apps at once, your computer struggles because it can't remember where everything is.

Remember this

Most computer tasks are memory-bound, meaning they're limited by how fast the computer can remember information, not by how fast it can calculate.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Strength reduction

Why do computers sometimes get tired of doing the same math over and over?

Attempts to overturn the 2020 United States presidential election

Why can computers do many things at once?

ReLU and Leaky ReLU

Why do computers sometimes struggle with simple decisions?

Loop nest optimization

Can speeding up your computer make tasks quicker?

2022 in science

Why do Transformers sometimes seem to 'ignore' irrelevant parts of the input?

Matrix multiplication algorithm

Ever wondered how computers speed up multiplying huge numbers?

Swipe through more Machine Learning concepts