NVIDIA has introduced NVHBM, a custom high‑bandwidth memory designed to pair with its NVLink Fusion ecosystem – enabling third‑party chip partners to build rack‑scale multi‑chip AI acceleration systems.

NVHBM is not a replacement for standard HBM, but an optimized companion for custom silicon. Compared with commercial HBM4e, it delivers:

• Up to 30% higher bandwidth per stack – directly boosting AI inference token throughput.

• 15% lower overall power – energy saved can be reallocated to extra compute or additional accelerators in large clusters.

Architecturally, NVHBM moves the memory controller onto the HBM base die (instead of the main compute die), freeing up to 30% of chip area for extra compute resources, while also simplifying interposer routing. It carries a custom, compact PHY that partners can integrate directly into their designs.

The first confirmed adopter is Amazon’s Annapurna Labs, whose next‑gen Trainium 4 AI chip already supports NVLink Fusion and is expected to adopt NVHBM – powering AWS’s cloud AI infrastructure.

NVIDIA notes that NVHBM is a future‑facing capability; the currently shipping Vera Rubin NVL72 racks do not yet use it.

ICgoodFind Takeaway:
NVHBM turns memory into a strategic differentiator – higher bandwidth, lower power, and more silicon real estate for compute. It’s a system‑level move that could accelerate custom AI silicon adoption.