Ethernet: The path to singularity

The AI revolution has completely reshaped the tech landscape, especially in cloud and enterprise data centers. The hunger for more processing power is impressive, but the real showstopper is the networking infrastructure needed to fuel AI operations.

Think of it this way: While traditional data center tasks can often be managed on a single server, AI training jobs are colossal. They require data to flow seamlessly between hundreds or even thousands of devices. Enter Ethernet, the hero, with its vast ecosystem and popularity in major data centers. It’s the go-to choice for handling AI workloads and has earned its stripes as the top dog in AI networking. Let’s zoom in on what makes Ethernet the ace for today’s AI jobs and why it’s destined to stay the gold standard for connecting all things computational.

AI traffic: All about east-west flow

Picture AI training as a bustling highway with data racing back and forth in an east-west fashion between endpoints connected to the highway. Very little traffic takes offramps to the north-south side streets outside of the data center. Over the past decade, data center traffic has been all about east-west, machine-to-machine communication, accounting for up to 90% of all data movement. Ethernet is tailor-made for this kind of traffic.

But AI has its quirks. Instead of many small data streams, it sends large “elephant flows” — sometimes megabytes in size. These beasts need special care to keep traffic running smoothly. Ethernet, starting with Broadcom switches as early as Tomahawk 2, saw this coming and implemented adaptive routing schemes including Dynamic Load Balancing (DLB) in 2016. DLB continuously analyzes outgoing links to make sure data takes the fastest route. Combined with other adaptive routing innovations, Ethernet switches are efficient, scalable, and up to the task of handling AI workloads.

The thriving AI ecosystem

The Ethernet scene is on fire, with exciting developments like Broadcom’s Jericho3-AI and 51.2Tbps Tomahawk 5, innovations like Linear Pluggable Optics (LPO), Microsoft’s acquisition of Fungible, and 51.2Tbps switches announced by Cisco, Marvell and NVIDIA. Not to mention the birth of Ultra Ethernet Consortium (UEC), entirely focused on enhancing Ethernet for next-gen AI networks. This is where the action is.

The power of community

UEC has taken the networking community by storm. Over 200 companies, organizations and institutions have expressed interest in joining since its launch in July 2023. The UEC Steering Members, including heavyweights like AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta, Microsoft and Oracle, are backed by decades of experience in networking, AI, cloud and high-performance computing. Together, they are driving Ethernet’s evolution to make it the go-to choice for large-scale AI networks.

One standout goal of UEC is the modernization of Remote Direct Memory Access (RDMA), a crucial technology for high-performance AI training. Traditional RDMA, used in today’s InfiniBand and Ethernet networks, has limitations. But the UEC’s Transport (UET) protocol introduces packet-level multipathing, out-of-order packet delivery with in-order message completion, efficient error handling, and configuration-free congestion control. This results in superior load balancing, better network utilization, and faster AI job completion times. It’s user and developer-friendly and designed to scale from a few thousand nodes to an impressive one million nodes.

Journey to the AI dream

The AI community’s ultimate goal is Artificial General Intelligence (AGI). Achieving this requires training on clusters of up to a million endpoints. Ethernet is up for the challenge, handling high bandwidth and managing massive congestion like a pro.

The consensus is clear: Top hyperscalers, founding members of UEC, and numerous new companies that will soon be joining UEC all rally behind Ethernet. It’s the bedrock of AI networks, propelling us towards the future of computing. In short, Ethernet is here to stay and will continue to shape the future of AI networks.