Authors
- Wenjie Hu National University of Defense Technology
- Sidun Liu National University of Defense Technology
- Peng Qiao National University of Defense Technology
- Zhenglun Sun National University of Defense Technology
- Yong Dou National University of Defense Technology
DOI:
https://doi.org/10.1609/aaai.v40i1.37003Abstract
Recent advances in Transformer-based Neural Operators have enabled significant progress in data-driven solvers for Partial Differential Equations (PDEs). Most current research has focused on reducing the quadratic complexity of attention to address the resulting low training and inference efficiency. Among these works, Transolver stands out as a representative method that introduces Physics-Attention to reduce computational costs. Physics-Attention projects grid points into slices for slice attention, then maps them back through deslicing. However, we observe that Physics-Attention can be reformulated as a special case of linear attention, and that the slice attention may even hurt the model performance. Based on these observations, we argue that its effectiveness primarily arises from the slice and deslice operations rather than interactions between slices. Building on this insight, we propose a two-step transformation to redesign Physics-Attention into a canonical linear attention, which we call Linear Attention Neural Operator (LinearNO). Our method achieves state-of-the-art performance on six standard PDE benchmarks, while reducing the number of parameters by an average of 40.0% and computational cost by 36.2%. Additionally, it delivers superior performance on two challenging, industrial-level datasets: AirfRANS and Shape-Net Car.
How to Cite
Hu, W., Liu, S., Qiao, P., Sun, Z., & Dou, Y. (2026). Transolver Is a Linear Transformer: Revisiting Physics-Attention Through the Lens of Linear Attention. Proceedings of the AAAI Conference on Artificial Intelligence, 40(1), 408–416. https://doi.org/10.1609/aaai.v40i1.37003
Issue
Section
AAAI Technical Track on Application Domains I