Paper 2026/160

Leveraging ASIC AI Chips for Homomorphic Encryption

Tianhao Huang, Massachusetts Institute of Technology

Jingtian Dang

Leo de Castro, Massachusetts Institute of Technology

Anirudh Itagi, Georgia Institute of Technology

anupam golder, Georgia Institute of Technology

asra ali, Google (United States)

Jeremy Kun, Google (United States)

jevin jiang, Google (United States)

arvind arvind, Massachusetts Institute of Technology

G. Edward Suh, Cornell University

Tushar Krishna, Georgia Institute of Technology

Abstract

Homomorphic Encryption (HE) provides strong data privacy for cloud services but at the cost of prohibitive computational overhead. While GPUs have emerged as a practical platform for accelerating HE, there remains an order-of-magnitude energy-efficiency gap compared to specialized (but expensive) HE ASICs. This paper explores an alternate direction: leveraging existing AI accelerators, like Google's TPUs with coarse-grained compute and memory architectures, to offer a path toward ASIC-level energy efficiency for HE. However, this architectural paradigm creates a fundamental mismatch with SoTA HE algorithms designed for GPUs. These algorithms rely heavily on: (1) high-precision (32-bit) integer arithmetic to now run on a TPU's low-throughput vector unit, leaving its high-throughput low-precision (8-bit) matrix engine (MXU) idle, and (2) fine-grained data permutations that are inefficient on the TPU's coarse-grained memory subsystem. Consequently, porting GPU-optimized HE libraries to TPUs results in severe resource under-utilization and performance degradation. To tackle above challenges, we introduce CROSS, a compiler framework that systematically transforms HE workloads to align with the TPU's architecture. CROSS makes two key contributions: (1) Basis-Aligned Transformation (BAT), a novel technique that converts high-precision modular arithmetic into dense, low-precision (INT8) matrix multiplications, unlocking and improving the utilization of TPU's MXU for HE, and (2) Memory-Aligned Transformation (MAT), which eliminates costly runtime data reordering by embedding reordering into compute kernels through offline parameter transformation. CROSS (TPU v6e) achieves higher throughput per watt on NTT and HE operators than WarpDrive, FIDESlib, FAB, HEAP, and Cheddar, establishing AI ASIC as the SotA efficient platform for HE operators. Code: https://github.com/EfficientPPML/CROSS

@misc{cryptoeprint:2026/160,
      author = {Jianming Tong and Tianhao Huang and Jingtian Dang and Leo de Castro and Anirudh Itagi and anupam golder and asra ali and Jeremy Kun and jevin jiang and arvind arvind and G. Edward Suh and Tushar Krishna},
      title = {Leveraging {ASIC} {AI} Chips for Homomorphic Encryption},
      howpublished = {Cryptology {ePrint} Archive, Paper 2026/160},
      year = {2026},
      url = {https://eprint.iacr.org/2026/160}
}