NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
Image: Hustvedt, CC BY-SA 3.0, via Wikimedia Commons
NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
NVLink provides: high-bandwidth GPU-to-GPU interconnect (900 GB/s on H100)
Related concepts
NVIDIA's H100 has: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16
NVIDIA H100 features: 80GB HBM3, 3.35TB/s bandwidth, 990 TFLOPS FP16
PCIe bandwidth limits: ~64 GB/s for PCIe 5.0 x16, bottleneck for CPU-GPU transfer
PCIe 5.0 x16 bandwidth limit ~64 GB/s, bottleneck for CPU-GPU transfer
NVIDIA's A100 has: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16
NVIDIA's A100 features: 80GB HBM2e, 2TB/s bandwidth, 312 TFLOPS FP16
HBM (High Bandwidth Memory) provides: stacked DRAM with much higher bandwidth than DDR
HBM provides stacked DRAM with much higher bandwidth than DDR
2024–present global memory supply shortage
Global DRAM shortage began in 2024
Dynamic random-access memory
DRAM requires periodic refreshing to maintain data integrity
Swipe through 100 ML concepts daily
Open Pocket Polymath