Microsoft Azure unveiled its brand new virtual machines on Monday that use NVIDIA’s H100 Tensor Core GPUs to accelerate generative AI, such as ChatGPT.
However, the interconnection of the GPUs will also depend on NVIDIA for its Quantum-2 InfiniBand networking. The ND H100 v5 will become a standard offering in Azure, but Azure didn’t confirm when.
Nvidia GPUs have been behind ChatGPT, and 10,000 Nvidia GPUs were used to instruct that model, as per the earlier reports that Microsoft also discussed in a blog on Monday. They also claim customers will see drastically faster performance for ND models over prior ND A100 v4 VMs. Further, Nvidia H100 GPUs are interconnected. Azure said there would be Quantum-2 CX7 Infiniband at 400 Gb/s per GPU as well as 4th Gen Intel Xeon Scalable processors.
Moreover, Nvidia said that its H100 Tensor core gives up to 9x faster AI training on huge models when compared to A100. For AI interference, the H100 is up to 30x faster.
According to Azure, “AI is swiftly becoming a persistent component of software and how we interact with it.” For Microsoft and organizations such as Inflection, Nvidia, and OpenAI that have devoted themselves to large-scale deployments, this contribution will allow for a new class of large-scale AI models.
However, these machines will make infrastructure that can grow to any size for any AI task.
