NVIDIA's A100 has: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16

NVIDIA's H100 has: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16

NVIDIA H100 features: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16

NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)

NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)

PCIe bandwidth limits: ~64 GB/s for PCIe 5.0 x16, bottleneck for CPU-GPU transfer

PCIe 5.0 x16 bandwidth limit ~64 GB/s, bottleneck for CPU-GPU transfer

HBM (High Bandwidth Memory) provides: stacked DRAM with much higher bandwidth than DDR

HBM provides stacked DRAM with much higher bandwidth than DDR

2024–present global memory supply shortage

Global DRAM shortage began in 2024

kernel fusion reduces memory bandwidth bottleneck

Can you imagine faster video games without waiting for loading screens?