Networking for Distributed AI: design principles, technologies, standards, and a sample product

Abstract

Distributed AI workloads are unique in the way they make use of the network. Consequently, traditional networking solutions are not ideal for interconnecting the massive number of processors deployed in clusters for AI training and inference. Given that the network plays a key role in distributed AI’s performance and power consumption, academia and industry have devoted an enormous amount of effort to develop specialized solutions.

After analyzing the unique networking requirements of distributed AI workloads, this talk describes the network architectures commonly used to achieve the extreme scale of AI clusters, and the standard protocol stacks developed for this specific context, covering both scale-out and scale-up domains. It will conclude by showing how all these aspects come together in a commercial solution.

Bio

Mario Baldi is a Distinguished Architect at NVIDIA, in the Networking Software and System Architecture group. A tenured professor for over two decades, he has also held various positions in startups and established companies in the computer networking industry, as well as several visiting professorships at universities on four continents, over more than 25 years of professional involvement in the field. He has authored over 150 scholarly papers on various networking-related topics and two books, and is an inventor on 36 patents.