South Korean semiconductor giant SK Hynix has announced a new type of high-bandwidth memory (HBM) for AI datacenters that improves heat dissipation by integrating a cooling layer within the memory package itself.

The change could allow AI processors incorporating the new memory to run faster, or reduce cooling costs.

Traditional chip cooling architectures are largely external; heat dissipation happens after it leaves the package. For the HBM memory used by AI, which vertically stacks memory chips on top of one another to improve latency and memory density, the extra heat generated has become a major design constraint.

Slated for the company’s next-generation HBM5 products due for launch from 2029 onwards, SK Hynix’s latest integrated high bandwidth memory (iHBM) takes a completely different approach of putting the cooling inside the Die-to-Die Physical Layer (D2D PHY).

This is the physical interface connecting the HBM and GPU where heat is concentrated. In iHBM this becomes a new ‘heat dissipation path’ for integrated cooling elements (ICE), reducing thermal resistance by a claimed 30%.

Not that long ago, innovations in memory and cooling would have been viewed as an interesting sideshow in a datacenter sector dominated by processor chip performance.

But as datacenter processor performance has grown rapidly over the last decade, the rise in importance of memory design, and the ability to cool it inside high-performance computing (HPC) systems, has turned into a big issue.

Made from custom silicon, putting ICE into memory packages makes life simpler for system builders. If iHBM can make good on the 30% improvement in heat dissipation that means the HBM modules have more headroom before hitting temperature ceilings that act as a drag on performance.

HBM boom

Memory’s importance to the AI datacenter boom is now so fundamental that recent figures from forecasting organization Epoch AI found that between Q1 2024 and Q4 2025 HBM rose from 52% to 63% of all AI chip component spending.

The numbers underline how AI has undermined decades of computing performance assumptions. With AI, the volume of data becomes critical and not simply the speed at which it can be processed. This has turned memory from an afterthought into something every datacenter architect worries about first.  By comparison with HBM, Epoch AI noted that logic dies — Nvidia’s famous GPUs, for example — fell slightly from 14.2% to 12.9% of spending over the same period.   

The knock-on effect of AI demand is that manufacturers have prioritized HBM over other types of memory such as DDR5, causing shortages for device makers.

In March, SK Group chairman Chey Tae-won said demand for hardware to run AI had overwhelmed supply in ways that looked like a longer-term structural change rather than a cyclical one. Epoch AI reckons this HBM demand boom has some way to go. “HBM will likely account for an even larger share in 2026 as memory supply remains tight and prices rise,” it said.

However, HBM is not the only show in town; in February Intel announced it was partnering with Softbank to develop an alternative, Z-Angle Memory (ZAM), also based on stacking memory modules on top of one another, with a delivery date of around 2030.

For AI datacenters designers and customers, every development is good news at a time when expectations for constantly rising performance have put the industry under pressure.

Improving thermal performance, and delivering it on time, could turn out to be a deciding factor. “iHBM is an optimal solution for thermal management, combining our memory design capabilities with advanced packaging technology,” said SK Hynix senior VP of PKG development, Kangwook Lee.