Given the need to reduce the latency between components and to cram more and more circuits into a socket for compute engines as well as network ASICs, it is inevitable that chip designers will moved out of the two dimensional world and start stacking up components.

We have gone vertical with DRAM memory with HBM stacks, which is relatively simple given the low power draw of memory chips compared to the ASICs that shuttle data around and compute upon it. We have so-called 2.5D stacking that is used on interposers to interconnect components like GPUs and XPUs to that HBM stacked memory, and AMD pioneered 3D stacking of L3 cache chips with its Epyc CPUs. Intel and AMD routinely use 3D stacking for cache memory on various CPUs and GPUs now, and I have always wondered why this has not become the norm given that it then allows for more compute cores to be put into a socket without cutting back on the cache.

The reasons we want to go vertical are intuitively obvious, just like it is obvious why the industry is building larger and larger sockets with 2.5D interconnects, creating what amounts to a virtual and larger 2D chip from multiple chiplets.

Plunking down four or eight GPUs or XPUs onto a system board has been normal for a decade or more in HPC and now AI systems. But interconnecting those compute engines with off-chip links – pick your poison here – burns somewhere between 3 picojoules per bit to 5 picojoules per bit, Harish Bharadwaj, the vice president of product marketing who is steering Broadcom’s 3.5D Extreme Dimension System in Package (XDSiP) chiplet stacking, tells The Next Platform.

If you collapse that system board with four compute engines down to a single socket, then it is less than 0.2 picojoules per bit to link the same compute elements using die-to-die links. There are obviously shorter distances staying inside the socket than using motherboard traces, which lowers latency as well as power. The resulting socket can be – and often is – scaled up further with a motherboard and high speed interconnects, so it is not like that is the end of it for system architects. But clearly, you want the highest performing socket you can make because that is the real unit of compute.

Which is why 3D stacking, no matter the complexities and costs, is inevitable. The typical 3.5D XPU that Broadcom is working on with customers has multiple stacked compute chiplets – not just one – and also has multiple stacks of HBM memory. The original 3.5D XDSiP topped out at a dozen HBM memory stacks, and Broadcom has been working to make that number even higher.

The reason, I surmise, is that XPU makers want to hang back on the HBM generation and use more of the cheaper HBM memory to get capacity and bandwidth. For instance, we have seen Google do this with its latest TPU 8 XPUs, which use HBM3E memory instead of the more current HBM4, and SambaNova Systems did it with its SN50 RDU, which uses HBM2E memory to keep it cheap and deep. (Google uses Broadcom to help chip shepherd the “Sunfish” TPU 8t through the TSMC foundry, but has not employed 3.5D XDSiP as far as we know.)

We do know that Fujitsu is, however, with its future “Monaka” Arm server CPU, which we did a deep dive on back in March 2023 and which we now know will have 144 Armv9-A cores using a mix of 2 nanometer and 5 nanometer chiplets. The Monaka chip has been manufactured in sample quantities and Fujitsu got them back from the Broadcom labs at the end of February after adding the 3D compute chip stacking to the Monaka design two years ago.

Here is what the Monaka sample looks like:

It is not clear exactly how Fujitsu will implement Broadcom’s 3.5D Extreme Dimension System in Package (XDSiP) chiplet stacking – the company is saving something to say for the launch of the Monaka chip when it comes out in 2027 – but Bharadwaj says that Fujitsu is stacking a compute chiplet using 2 nanometer processes atop another compute tile with 5 nanometer processes.

There are a half dozen other companies implementing 3.5D XDSiP in their custom AI XPU designs, says Bharadwaj. Two out of the six XPUs makers should be Amazon Web Services with its Trainium4 due at the end of this year but probably installing in volume in 2027 and Meta Platforms with its MTIA 500 also due in 2027. But that is just conjecture.

“The key thing is that customers using 3.5D XDSiP is to keep the top die in the most advanced silicon node so that it can do the highest performance compute,” Bharadwaj explains. “There are customers doing 3 nanometer over 3 nanometer, 2 nanometer over 3 nanometer, and even 1.4 nanometer over 3 nanometer. That thing is kind of evolving. The point is, putting the high performance compute at the top makes it easier for the heat to escape, and then you put the SRAM and some low activity compute and the interconnect at the bottom so that the heat is less and but is still able to escape.”

Bharadwaj says that Broadcom has been working on 3.5D XDSiP for over five years, and that it is a different approach to the “face-to-back” 3D SoIC approach that AMD created and developed with Taiwan Semiconductor Manufacturing Co and has used, for instance, to stack L3 dies atop compute dies and interlink them with pins on the chip.

Here is what the face-to-back 3D SoIC approach looks like:

Keep your eye on that TSV density. With face to back approaches thus far, Bharadwaj says that you can get a signal density of maybe 1,500 signals per square millimeter, and that means chip designers have to be mindful of the architecture of the top and bottom chips and how they link together.

If you go face-to-face on the chip stacking, the metal on the two dies are already aligned, and you don’t have to do anything special in the 2D chiplet designs to make that happen. All you need is a bonding agent to make them stay linked, which Broadcom and TSMC have been working on together to develop 3.5D XDSiP. Like this:

With 3.5D XDSiP, the signal density between the two chips is almost an order of magnitude larger, with 14,000 signals per millimeter squared.

And that is why there is one CPU and six XPUs lined up to use the technology. Fujitsu will not be the first to ship, but at least one of the six will ship sometime in the second half of 2026 according to Broadcom.