Lambda says its 10,000-GPU Nvidia cluster in Quebec is the first large-scale production deployment of co-packaged optics

AI & Compute

Lambda's 10,000-GPU Quebec Cluster Runs First Large-Scale Co-Packaged Optics

Lambda says its Quebec cluster of 10,000+ Nvidia Blackwell Ultra GPUs is the first large-scale production deployment of co-packaged optics, with measured latency gains.

By
Nathan Brooks
Filed
Channel
AI & Compute
Read
3 min read

Lambda has connected more than 10,000 Nvidia Blackwell Ultra GPUs at its Quebec campus through Nvidia Photonics co-packaged optics switches, and it calls the result the world's first large-scale production CPO deployment. Nvidia said on Monday, Nov. 16, that the GB300 NVL72 racks in the cluster are linked by the switches, and it is featuring the deployment in its special address at SC26, the supercomputing conference.

The claim carries a date that matters. ServeTheHome, which has covered CPO for years, reported the cluster is running today and called it the first production CPO cluster — a few weeks after a vendor keynote at the OCP Summit described CPO as still years away. Lambda first announced in March, at Nvidia's GTC conference, that it was leading one of the largest deployments of Nvidia's Quantum-X InfiniBand Photonics CPO switches in a facility with more than 10,000 GPUs. Monday's release adds the first performance data from that hardware.

What does co-packaged optics actually change?

In a conventional AI network, the switch chip sends electrical signals over copper to pluggable optical transceivers mounted in cages on the front panel, where they become light for the fiber. CPO moves the optics onto the switch package itself. High-speed signals no longer travel across that copper path, which cuts power — power that can go to GPUs instead.

Each switch in the Lambda deployment offers 144 ports at 800Gb/s and is liquid cooled. Light comes from 18 removable external laser sources on the front of the chassis, swappable in the field. ServeTheHome noted that lasers are the parts that tend to fail in pluggable optics, and 18 light sources replace what would otherwise be 144 800Gb/s transceivers. Fewer lasers, easier cooling, fewer failures. The first version of Nvidia's CPO packaging sits in its Quantum-X InfiniBand switches; Spectrum-X Ethernet will get a second-generation part.

What did Lambda measure?

Lambda tested CPO against conventional pluggable optics at rack, scalable-unit and pod scale. Throughput matched. Signal integrity was more stable. Latency fell at every scale, with the gap widening from 3.8% at rack scale to 10.1% at pod scale.

Median optics temperature ran at 36 degrees C, against 51 degrees C for pluggable optics — roughly the difference between a warm server room and a hot one. Lambda says the cooler optics deliver up to 15x higher reliability. Nvidia's vendor figures for its photonics switches claim up to 5x better power efficiency, up to 15x higher reliability and 4x fewer lasers than pluggable optics. Those are vendor numbers, not Lambda failure data. Neither company published failure rates or a network power figure for the Quebec cluster.

Who is behind the cluster?

"Networking has to keep pace with the demands of increasingly large AI clusters," said Dave Ward, Lambda's president of cloud services. Lambda says the Quebec campus holds 142 MW of secured, nearly 100% renewable capacity and a guaranteed power usage effectiveness of 1.2.

The commercial backdrop is less tidy. Lambda is a major supplier of compute to Anthropic, and Anthropic's amended S-1, filed Nov. 10, says its roughly $35 billion August agreement with Lambda can now be terminated by either party because Lambda did not secure the facility lease. Neither side has given notice. If Anthropic walks, Lambda will have a very large, very efficient network looking for a customer.

Source: mlq.ai

Share this article:

More from Nathan Brooks

Nathan Brooks

Show full bio

Senior reporter covering people, launches and day-to-day practice in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,012 articles

Related articles

« Previous article