






















Andy Bechtolsheim, legendary co-founder of Unix system maker Sun Microsystems and more than a few networking startups after that, has nothing against co-packaged optics. Well, except that thus far CPO modules have not been able to be manufactured in volume.
There is no question that co-packaged optics is coming to datacenters at this point, particularly with AI industry juggernaut Nvidia embracing CPO links on its Quantum X800 InfiniBand and Spectrum X800 Ethernet switches – previewed in June 2024, launched in March 2025 and shipping since December 2025. Nvidia is also told customers in March this year at GTC that with the future “Feynman” GPUs coming in 2028 that the NVSwitch 8 coherent memory scale up network at the heart of its rackscale systems will be moving to CPO as well. This strongly implies that the Feynman GPU and whatever the next generation of Arm server CPU that Nvidia pairs with it – we don’t have a codename as yet – will also have CPO ports.
NEXTPLATFORM AD
To address the manufacturability issues, ahead of GTC this year, Nvidia pumped $2 billion each into Lumentum and Coherent, who among other things make the lasers that drive CPO ports. Nvidia also inked multi-billion dollar, multi-year supply agreements with these companies. And later in March, Nvidia inked a similar $2 billion deal with Marvell to add NVLink Fusion ports to custom accelerators, but we think there is a chance that some CPO technology from the $2.5 billion Celestial AI acquisition by Marvell back in December 2025 is also part of this deal.
In the meantime, AI datacenters need more density in their networks and they need optics that are denser than the SFP, QSFP, and OSFP pluggable modules that have dominated the datacenter for the past two decades. And more specifically, they need more density than the OSFP pluggables that were first conceived in 2016 by Arista Networks, the upstart who took on Cisco Systems in datacenter networking where Bechtolsheim is co-founder and chief development officer, and Google and the industry adopted the standard and delivered pluggables at 400 Gb/sec speeds a year or so later. These have become the most popular pluggable optics in history.
The trouble is, the OSFP modules are too fat for the radix that is necessary for modern AI systems, particularly if you want to use Ethernet for scale up and scale out. Enter Extra-dense Pluggable Optics multi-source agreement, which is a standard started by Arista Networks, Microsoft, Marvell, Broadcom, and Ciena that now has over a hundred companies backing it. Google is not on the list, and that might mean that Google is going to try to jump from OSFP pluggables to some sort of on-chip CPO in the same timeframe that XPO modules are expected to come to market in volume.
The XPO module is clever in that it packs a whole lot more bandwidth into the same space as an OSFP module, and that means the front panels on switches have a lot more radix coming out of them as well as more bandwidth. Because this is physics we are dealing with, you have to pay for the XPO module in heat density, and that means the XPO module has to remove that heat with a cold plate and liquid cooling. But this is a minor thing in a world where GPUs and XPUs need to be connected in larger and larger scale up and scale out domains and liquid cooling is becoming normal for rackscale systems. There is no other way to get components closer to each other and therefore drive down latency and drive up performance.
NEXTPLATFORM AD
According to Bechtolsheim, right now, using 1.6 Tb/sec OSFP modules, you can get 32 ports for a total of 51.2 Tb/sec coming out of the front panel of a 1U Ethernet switch. The OSFP modules burn somewhere between 30 watts and 40 watts, and if you slap a cold plate on it, you can’t really increase the cooling capacity or the switch front panel port density. What that means is if you have a 204.8 Tb/sec switch ASIC, which believe it or not we will have soon, you need 4U of chassis space to get 128 OSFP modules running at 1.6 Tb/sec.
The XPO module crams 64 channels running at 200 Gb/sec into the same space taken up by two OSFP modules, which increases lane density by a factor of four. Bandwidth goes up by a factor of eight with the XPO module, and heat dissipation goes up by a factor of ten to 400 watts.
Here is the exploded view of the XPO module:
There is some lucky geometry going on here, says Bechtolsheim, in that the paddle cards – the little motherboards of circuits – fit into the same exact space as two OSFP modules. So the chips and paddle card designs did not have to be changed. You put two side by side and then stack a pair of those pairs belly to belly and you get eight times the lanes in twice the space.
NEXTPLATFORM AD
When you look at it, XPO seems kind of intuitively obvious. Like many good engineering ideas do in hindsight.
The XPO module will support a variety of front panel fiber connectors:
The upshot, says Bechtolsheim, is that XPO supports any optics standard, any optics technology, any type of driver, retimer, or gearbox, any optical connector, and any kind of cable, and it can give density improvements without having to shift to CPO. Presumably the economics will be better, but don’t jump to that conclusion too fast.
One other side benefit is that with liquid cooling on the XPO components, they run anywhere from 20 degrees to 25 degrees cooler (that’s in Celsius) in a 12.8 Tb/sec ZR module, with its schematic and heat map shown below, than an air-cooled 1.6 Tb/sec OSFP-ZR module. (That is around 45 degrees Celsius for the XPO module compared to 65 degrees to 70 degrees for the OSFP module at the same 1.6 Tb/sec bandwidth.)
NEXTPLATFORM AD
That lower temperature means fewer failures in the field – how much remains to be seen. But this has real monetary value in an AI supercomputer where any outage stops a training run cold and you have to roll back to a checkpoint and start again. Time is money when it comes to GPUs and XPUs, and a lot of money at that.
In the switch designs that will come out using XPO, the power for the XPO modules will be drawn directly off of a 50 volt bus bar, rather than going through motherboard voltage converters used in switches using OSFP modules. This is a more efficient way to distribute the power.
The net-net is that a rack of switches using XPU modules can cram an aggregate of 6.5 Pb/sec into an Open Rack v3 switch rack from the Open Compute Project, which looks like this:
But here is the money math that will have AI datacenter builders interested in the XPO modules. They can cut the size of their datacenters in half because of the increase in density of the network racks. The change is dramatic.
Let’s say that you want to have a row of compute and networking based on Ethernet that links 512 XPUs together and that you want to use switches with top-of-the-line 1.6 Tb/sec OSFP ports. You will have 128 compute engines per rack, and need four racks for the compute, but it will take eight racks of networking. With a shift to XPO, you have the same four racks of XPUs, but you only need two racks of switching. So twelve racks collapses down to six racks for the same compute and the same interconnectivity. The switch racks are hotter, but there are fewer of them and the power is mostly a wash. The lengths of the cables between the XPUs and the switches are also shorter, which is a benefit in that it is less fiber, which costs money. Over a 1 gigawatt datacenter, such little things add up, as does having to pour less concrete and putting up smaller shells for a given amount of compute and networking.
So how far does XPO push out CPO? And Bechtolsheim gave us the same answer he has given for years now.
“We have told both customers and said in public, we are not religious about any technology here,” Bechtolsheim tells The Next Platform. “So that is understood. The only thing we are religious about is that we can ship stuff in high volume. Everybody working on XPO did their own work based on their own nickel, and they all want to own what they developed, and they all are going to be there. This effort has been driven by this one large end customer, but I think everybody looked at this and concluded that this the way to get the next level of density.”
There are more than 20 different vendors that are going to be making XPO modules, and they are expected to be in volume production in 2027.
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。