NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
Image: Hustvedt, CC BY-SA 3.0, via Wikimedia Commons
NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
Related concepts
NVIDIA's H100 has: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16
NVIDIA H100 features: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16
PCIe bandwidth limits: ~64 GB/s for PCIe 5.0 x16, bottleneck for CPU-GPU transfer
PCIe 5.0 x16 bandwidth limit ~64 GB/s, bottleneck for CPU-GPU transfer
NVIDIA's A100 has: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16
NVIDIA's A100 features: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16
HBM (High Bandwidth Memory) provides: stacked DRAM with much higher bandwidth than DDR
Ever wondered how your computer speeds up?
2024–present global memory supply shortage
Global DRAM shortage began in 2024
Dynamic random-access memory
DRAM requires periodic refreshing to maintain data integrity
Swipe through 100 ML concepts daily
Open Pocket Polymath