Nvidia isn’t the only company that makes AI accelerators for training and inference, it’s a space that Intel is aggressively competing and excelling in too with its Intel Gaudi 2 technology, according to new research. Databricks conducted new research that is being released today, revealing that Intel Gaudi 2 provides strong performance competition against the industry-leading AI accelerators from Nvidia. The Databricks research found that for large language model (LLM) inference, Gaudi 2 matched the latency of Nvidia H100 systems on decoding and outperformed the Nvidia A100. The research found that Gaudi 2 inference achieves higher memory bandwidth utilization than H100 and A100. Nvidia still provides more training performance on its top-end accelerators. Using the Databricks MosaicML LLM foundry for training, the researchers found that Gaudi 2 achieved the second fastest single-node LLM training performance after NVIDIA H100, with more than 260 TFLOPS/chip. Overall, the Databricks research reported that based on public cloud pricing, Gaud i2 has the best dollar-per-performance for both training and inference compared to A100 and H100. Intel has been providing its own testing results on Gaudi 2 via the MLcommons MLperf benchmark for both training and inference. The new data from Databricks provides further validation for Intel on the performance of its Gaudi technology, from a third party.
Full research : Databricks research confirms that Intel’s Gaudi bests Nvidia on price performance for AI accelerators.
