Nvidia is introducing a new top-of-the-line chip for AI work, the HGX H200. The new GPU upgrades the wildly in demand H100 with 1.4x more memory bandwidth and 1.8x more memory capacity, improving its ability to handle intensive generative AI work. The big question is whether companies will be able to get their hands on the new chips or whether they’ll be as supply constrained as the H100 — and Nvidia doesn’t quite have an answer for that. The first H200 chips will be released in the second quarter of 2024, and Nvidia says it’s working with “global system manufacturers and cloud service providers” to make them available. Nvidia spokesperson Kristin Uchiyama declined to comment on production numbers. The H200 appears to be substantially the same as the H100 outside of its memory. But the changes to its memory make for a meaningful upgrade. The new GPU is the first to use a new, faster memory spec called HBM3e. That brings the GPU’s memory bandwidth to 4.8 terabytes per second, up from 3.35 terabytes per second on the H100, and its total memory capacity to 141GB up from the 80GB of its predecessor. “The integration of faster and more extensive HBM memory serves to accelerate performance across computationally demanding tasks including generative AI models and [high-performance computing] applications while optimizing GPU utilization and efficiency,” Ian Buck, Nvidia’s VP of high-performance computing products, said in a video presentation this morning.
Full story : Nvidia launches H200 a new must-have AI chip — as customers still scramble for its last one.
