OpenAI’s New Fix for A.I.’s Hidden Bottleneck

A New Bid to Fix the Hidden Chokepoint in A.I.

OpenAI on Wednesday introduced a new networking protocol designed to address one of the less visible but increasingly consequential problems in artificial intelligence: how to move enormous volumes of data fast enough, and reliably enough, across giant clusters of chips training the world’s most advanced models.

The protocol, called Multipath Reliable Connection, or MRC, was developed with AMD, Broadcom, Intel, Microsoft and Nvidia, and has been released as an open specification through the Open Compute Project, the industry consortium that has become a key venue for sharing data center hardware designs.

OpenAI said the technology is already running in some of its largest Nvidia GB200-based training systems, including installations in Abilene, Texas, built with Oracle Cloud Infrastructure, and in Microsoft’s Fairwater supercomputers. The company said the protocol has been used in production training runs for recent frontier models, including systems behind ChatGPT and Codex.

The announcement offers a more concrete look at a costly engineering challenge that has become central to the race to build ever larger A.I. systems. Training a leading model is no longer just a matter of assembling more graphics processing units, or GPUs. It also requires ensuring that tens of thousands, and increasingly more than 100,000, chips can exchange data without delays, congestion or failures that can stall an entire synchronized workload.

Why the Network Has Become the Problem

As A.I. models have grown larger, the computers used to train them have expanded into sprawling clusters spread across vast data halls. In those systems, performance is often limited not by the speed of individual chips but by the network connecting them.

Modern training runs depend on tightly synchronized communication among GPUs. If one path in the network slows down or a switch fails, the effects can ripple across the entire job. Engineers sometimes describe this as a tail-latency problem: the slowest packets, not the average ones, can determine the pace of the system.

OpenAI and its partners say MRC is meant to reduce those bottlenecks by breaking transfers across hundreds of network paths at once, instead of relying on a smaller set of routes that can become congested or vulnerable to failures. If a path goes down, traffic can be rerouted in microseconds, the company said, with less disruption to training.

That design, OpenAI said, also makes it possible to build clusters of more than 100,000 GPUs using just two tiers of Ethernet switches rather than the three- or four-layer topologies often required in very large deployments. Fewer switch layers can mean lower latency, simpler operations, and reductions in both power consumption and equipment costs, especially for switches and optical links that have become expensive elements of A.I. infrastructure.

An Ethernet Push in the Supercomputer Era

The new protocol is an extension of RoCE, or RDMA over Converged Ethernet, a technology used to enable low-latency, high-throughput communication over Ethernet networks. OpenAI said MRC also draws on work from the Ultra Ethernet Consortium and uses SRv6-based source routing, a method for steering traffic along explicit paths through a network.

Those details matter because the contest over A.I. infrastructure is not just about chips but about standards. Nvidia’s systems have been dominant in large-scale A.I. training, and its networking products have given it a powerful position in how these clusters are built. By advancing an open Ethernet-based protocol with a broad group of hardware and cloud partners, OpenAI is aligning itself with a larger industry effort to make hyperscale A.I. systems more interoperable and less dependent on proprietary interconnect approaches.

That does not mean the technical debate is settled. One of the open questions is whether MRC will gain traction beyond OpenAI’s immediate ecosystem, or whether it will remain a solution tailored mainly for a handful of frontier laboratories and cloud providers operating at exceptional scale. Another is how quickly switch makers, network interface card vendors and system builders can deliver interoperable implementations across products.

Why This Matters Now

The timing reflects the economics of the current A.I. boom. As companies pour tens of billions of dollars into data centers, small efficiency gains in networking can translate into large savings in power, hardware and uptime. They can also determine whether a company can practically train the next generation of models.

OpenAI has been expanding its infrastructure footprint as demand for model training and inference rises, and the release of MRC suggests the company is trying not only to secure more compute but to shape the design of the systems that will power future models. Contributing the specification through the Open Compute Project fits a broader pattern in which major A.I. players open parts of the infrastructure stack to encourage wider ecosystem support, even while competing fiercely at the model layer.

For outsiders, networking protocols can seem like an obscure corner of the A.I. race. But in frontier model development, the plumbing increasingly determines the pace of progress. The ability to keep massive GPU fleets communicating efficiently may be as important as securing the chips themselves. OpenAI’s latest move is a reminder that the next leap in A.I. may depend not just on smarter models, but on faster, sturdier ways to connect the machines that train them.

Sources

Further reading and reporting used to add context:

AI News