Nvidia-backed photonics startup Ayar Labs eyes hyperscale customers with GUC design collab
Who needs 600 kilowatt racks when a single computer can span entire datacenters?
EXCLUSIVE Nvidia-backed photonics startup Ayar Labs announced a new collaboration with Global Unichip Corp (GUC) on Sunday to integrate its optical I/O chiplets into the Taiwanese semiconductor design services provider's XPU reference designs.
The idea is that rather than using copper interconnects, which have limited reach and often require the use of costly retimers, Ayar's TeraPHY photonic chiplets can be co-packaged alongside compute dies, enabling more than 200 Tbps of aggregate bandwidth for chip-to-chip communications.
If Ayar and GUC can deliver on this design point, that'd make XPUs based on the tech more than 10 times faster than the fastest interconnects today, which top out at 14.4 Tbps.
This kind of optical I/O could eventually allow compute domains to scale beyond one rack to many, or even across entire aisles or data halls, without compromising on power consumption.
"We want to be able to scale up to 10,000 GPU dies connected in a scale-up domain, while keeping the rack power and power density to around 100kW," Ayar CTO Vladimir Stojanovic told El Reg.
You may recall that one of the reasons that Nvidia opted for copper cabling in its NVL72 rack systems was that the optical transceivers available at the time for converting electrical signals to light and back again would have added another 20 kilowatts to a system already rated for 120kW of capacity.
These high-speed interconnects allow the system's 18 compute blades to function as one big GPU. But relying on copper means those accelerators need to be in the same rack. At the speeds the interconnects operate at, the signals begin to degrade over about a meter.
If you were wondering why Nvidia is moving toward 600kW racks, it's because they're expecting to use copper to stitch the machine's 576 GPUs together into one great big compute domain.
By integrating optics directly into GPUs and other AI ASICs, chip designers can scale beyond the limits of copper to form even bigger domains and spread those chips over a larger area without compromising their performance. No need to build a 600kW rack because the GPUs no longer need to be packed so closely together. Or at least that's the idea.
Ayar has integrated its optical I/O into multiple prototypes, including one built in collaboration with Intel and the Defense Advanced Research Projects Agency (DARPA).
"The technology works, we've demonstrated that; the form factor works. It's a chiplet. It's integratable into the whole multi-chip package platform," Stojanovic said.
However, validating its tech for use with TSMC-fabbed chiplets in production environments is a different matter entirely. If an optical I/O die fails, it could turn a $50,000 accelerator into a paperweight.
That's where Ayar's collaboration with GUC comes into play. The design services provider will take Ayar's designs and validate them against a number of reference architectures on which hyperscalers can eventually base future designs.
"With GUC, we're working to create a reference flow that high volume manufacturing customers can access," Stojanovic said.
It's an open secret that many hyperscalers often license intellectual property from the likes of Broadcom, Marvell, and others rather than dedicating internal resources to reinventing the wheel. In theory, Ayar's work with GUC should provide hyperscalers with a shortcut to integrating photonics into future designs.
"Our new joint design allows us to address the challenges of CPO integration – architectural, power and signal integrity, mechanical and thermal – ensuring our future customers have access to a robust, high-bandwidth and power-efficient solution," Igor Elkanovich, CTO of GUC, said in a canned statement.
Initially, Ayar and GUC will be targeting XPU designs using multi-chip packages using the UCIe-S and UCIe-A chiplet interconnect standards, for package-level and die-to-die integration of its optical I/O dies, respectively.
While this is a step toward photonic accelerators, don't expect the collaboration to bear fruit immediately. Stojanovic expects that sometime in the next two years, the technologies will mature enough that we see optical I/O integrated into accelerators in volume. ®