NVIDIA's H100 has: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16

NVIDIA's A100 has: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16

NVIDIA's A100 features: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16

NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)

NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)

HBM (High Bandwidth Memory) provides: stacked DRAM with much higher bandwidth than DDR

HBM provides stacked DRAM with much higher bandwidth than DDR

PCIe bandwidth limits: ~64 GB/s for PCIe 5.0 x16, bottleneck for CPU-GPU transfer

PCIe 5.0 x16 bandwidth limit ~64 GB/s, bottleneck for CPU-GPU transfer

2024–present global memory supply shortage

Global DRAM shortage began in 2024

kernel fusion reduces memory bandwidth bottleneck

Can you imagine faster video games without waiting for loading screens?