AI & ComputeLOT 261120 · 3 min read
100,000+ accelerator clusters push networking beyond one data center
AI clusters above 100,000 accelerators need links across campuses and regions. Google has introduced Virto, while Cisco and Marvell develop related scale-across hardware.
Key takeaways
- Modern AI clusters typically accommodate more than 100,000 accelerators and potentially millions of CPU endpoints.
- Google introduced the Virto network architecture earlier in 2026 for workloads beyond single-data-center space and power limits.
- Cisco and Marvell are developing high-capacity routers, switching chips and optical interconnects for geographically distributed clusters.
- New Ethernet generations offer nominal bandwidths of 800 Gb/s and 1.6 Tb/s, but peak rates do not guarantee application performance.
- The source names libfabric, MPI, UCX, GPUDirect RDMA, HPE Slingshot and Nvidia SHARP as important networking layers.

By Sophie Lindqvist
Modern AI clusters typically accommodate more than 100,000 AI accelerators and potentially millions of CPU endpoints. That scale already strains bandwidth, latency management and congestion control.
At Supercomputing 2026 (SC26), researchers from The Ohio State University explained why accelerator performance now matters less without matching communication capability. “Faster AI GPUs do not necessarily deliver proportional improvements in application performance,” the researchers said. Fabric efficiency determines how much installed compute can work as intended.
What must connect at 100,000 accelerators?
High-performance computing has long divided networking into two domains. Scale-up and scale-out serve different distance, bandwidth, latency and scalability requirements.
- Scale-up: AMD Infinity Fabric, Intel UPI, Nvidia NVLink and the emerging UALink connect processors and accelerators within tightly integrated systems.
- Scale-out: Ethernet, InfiniBand, HPE Slingshot and the newer Ultra Ethernet connect servers or rack-scale systems into larger clusters.
At cluster scale, the network must feed compute without turning congestion, packet loss or synchronization overhead into system-wide delays. Raw accelerator throughput cannot compensate for those failures.
Why move computation across data centers?
Scale-across extends high-performance connectivity among buildings, campuses and regions. Operators can pool resources across sites instead of placing an excessive number of accelerators in one facility constrained by power availability, cooling requirements or physical space.
Google introduced its Virgo network architecture earlier in 2026 for AI workloads beyond one data center’s space and power limits. The design treats an entire computing campus as one system, while retaining specialized networks for communication among accelerators, servers and data centers.
Cisco and Marvell are developing high-capacity routers, switching chips and optical interconnects for geographically distributed clusters. These efforts target the same commercial requirement, but the announced development work included no capacity commitments, shipment schedules or pricing.
Can separate sites behave as one system?
Geography does not disappear when two clusters gain a high-speed link. Even optical networks cannot remove propagation delay. Congestion, packet loss and synchronization overhead further reduce effective performance, so connected GPU clusters need not behave like one tightly integrated machine.
AI training makes that distinction especially costly. Accelerators exchange model parameters, gradients and intermediate results through collective communications. Delays affecting one accelerator can reduce performance across the entire cluster.
Distributed performance therefore depends on more than link speed. Network topology, congestion management, communication protocols and software implementations all shape the result. No single faster interconnect addresses every failure mode.
What can hardware offload?
Network interface controllers can perform Remote Direct Memory Access operations without continuous CPU involvement. HPE Slingshot hardware collectives and Nvidia SHARP can offload selected communication tasks to network infrastructure. SmartNICs and data processing units move additional network and infrastructure functions away from host processors.
These capabilities can improve distributed application efficiency without increasing physical-link speeds. They do not remove the physics of distance or automatically prevent congestion.
Software carries equal weight. Communication frameworks such as libfabric, MPI and UCX determine how applications exchange data and use modern network adapters. GPU-aware communication and GPUDirect RDMA can remove unnecessary transfers through host memory, while optimized collective operations can reduce overhead across large accelerator counts, according to the researchers.
Will 1.6 Tb/s Ethernet solve the bottleneck?
The industry continues to introduce Ethernet versions with nominal bandwidths of 800 Gb/s and 1.6 Tb/s. Those figures describe link peaks, not guaranteed application throughput.
“Peak transfer rates alone do not fully describe the capabilities of a given system,” the researchers said. Actual performance depends on how network hardware and software cooperate to move data, prevent congestion and synchronize computations across thousands of processors.
The next generation of AI systems will depend on more than faster CPUs and accelerators. Sophisticated network adapters, switches and collective-communication software must unite distributed resources without excessive overhead.
Google’s Virto introduction shows that scale-across implementation has begun. Cisco and Marvell indicate where product development is heading. As clusters cross facility boundaries, usable performance will depend as much on networking execution as on accelerator speed, leaving cost and deployment scale as the unresolved commercial questions.
Tom's Hardware (Source)
More from Sophie Lindqvist
Show full bio
News editor covering money, funding and market shifts around Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.
1,194 articles