Matrix multiplication algorithm

Ever wondered how computers speed up multiplying huge numbers?

Imagine you're organizing a huge party and need to quickly calculate how many gift bags to prepare for all guests.

Picture each gift bag as a square and guests as numbers. Instead of counting each guest individually, you group them into smaller teams (blocks) and multiply them together first. This way, you handle fewer numbers at a time, speeding up the process.

Example

If you have 4 teams of 4 guests each, instead of multiplying 4 guests by 4 guests for each team, you multiply 2 guests by 2 guests for each team (2x2 = 4) and then multiply the results (4x4 = 16).

Remember this

Grouping numbers into blocks reduces the total number of calculations needed.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

tensor cores are

Why can computers crunch numbers faster than humans?

General-purpose computing on graphics processing units

Did you know your computer can do more than just compute numbers?

Computational complexity of matrix multiplication

O(n³) naive matrix multiplication

Overlapping subproblems

Ever calculated a huge Fibonacci sequence by hand?

most transformer operations are memory-bound, not compute-bound

Why do computers sometimes get tired?

tensor cores do 4x4 matrix multiply in one clock cycle

Tensor cores perform 4x4 matrix multiply using optimized GEMM (General Matrix Multiply) instructions in one clock cycle

Swipe through 100 ML concepts daily

Open Pocket Polymath