Paper 2026/198

ELLMo: Packing- and Depth-Aware Encrypted Transformer Inference

Lohit Daksha, Boston University

Carlos Agulló Domingo, University of Murcia

Gilbert Jonatan, Korea Advanced Institute of Science and Technology

John Kim, Korea Advanced Institute of Science and Technology

Jose L. Abellan, University of Murcia

David Kaeli, Northeastern University

Ajay Joshi, Boston University

Abstract

Cloud-based Large Language Model (LLM) inference processes sensitive user inputs, yet current deployments offer limited confidentiality guarantees. Fully Homomorphic Encryption (FHE) can provide strong privacy, but it clashes with transformer architectures, where rigid ciphertext packing demands expensive rotations, and deep polynomial circuits for nonlinearities necessitate costly bootstrapping. Although recent work has reported promising speed-ups, maintaining model accuracy is a challenge. We address these issues with ELLMo, a packing- and depth-aware encrypted transformer design. ELLMo introduces a novel matrix multiplication algorithm to reduce the ciphertext rotations. Further, head-split and merge steps are fused into this new algorithm at no additional cost. To reduce the depth of nonlinear layers, our contributions, Statistical-max Softmax and DelayNorm, help bypass deep comparison trees and homomorphic divisions to reduce bootstrapping by up to 46%. On encrypted BERT-Tiny, ELLMo achieves a $1.4\times$ speedup over state-of-the-art baselines with 0-1.5% accuracy loss across SST-2, MRPC, and RTE downstream tasks.

@misc{cryptoeprint:2026/198,
      author = {Seyda Nur Guzelhan and Lohit Daksha and Carlos Agulló Domingo and Gilbert Jonatan and John Kim and Jose L. Abellan and David Kaeli and Ajay Joshi},
      title = {{ELLMo}: Packing- and Depth-Aware Encrypted Transformer Inference},
      howpublished = {Cryptology {ePrint} Archive, Paper 2026/198},
      year = {2026},
      url = {https://eprint.iacr.org/2026/198}
}