NVIDIA Introduces New High-End Chip for Training AI Models

NVIDIA unveiled its newest high-end chip called H200 which is designed to train and deploy various kinds of AI models.

All About NVIDIA's H200 GPU

The new graphics processing unit (GPU) is an upgraded version of the H100, the chip that is used by OpenAI to train its most advanced large language model, GPT-4. The H100 chips are estimated to cost between $25,000 and $40,000 and companies need thousands of chips in order to develop and train their biggest models.

NVIDIA shared that the H200 performance is twice as fast as its predecessor when tested on Meta's Llama 2 LLM. Moreover, the chip is also equipped with 141GB of next-generation HBM3 memory that helps the chip perform inference. It will be available in four-GPU or eight-GPU server configurations on the company's HGX complete system.

The H200 will reportedly ship by the second quarter of 2024. It is also predicted to compete with AMD's MI3000X GPU which offers additional memory compared to previous models. Meanwhile, the H200 will still be compatible with the H100, this will allow AI companies to use the new version without changing their server systems.

Big Companies Fight for Limited Chip Supply

With the boom of generative AI, several companies and government agencies have been targeting to secure their supply of chips to train their models. NVIDIA's new AI GPUs boosted the company's stock by over 230% in 2023. For its third quarter, the company is expecting at least $16 billion in revenue.

Last October, NVIDIA revealed that the company will be following a one-year release pattern due to the high demand for GPUs. In addition, it was also revealed that tech companies can expect the release of the B100 chip, a GPU based on Blackwell architecture, in 2024. Previously, NVIDIA has been doing two-year architecture releases.