Abstract:Neural operators perform well on structured domains, yet their behaviour on irregular geometries remains poorly understood. We show that this limitation is not merely an encoding issue, but a depth-wise failure mode inherent to deep operator architectures.
We formalise the Geometric Forgetting Hypothesis: due to the Markovian structure of operator layers and their reliance on global mixing mechanisms, neural operators progressively lose access to domain geometry as depth increases. Using layer-wise geometric probing, we demonstrate that both spectral and attention-based operators systematically lose geometric fidelity.
We show that this geometric forgetting degrades accuracy, stability, and generalisation. To counteract it, we introduce a lightweight geometry memory injection mechanism that restores geometric constraints at intermediate depths with minimal architectural overhead. This simple intervention consistently mitigates forgetting and exposes a geometric shortcut instability in transformer-based operators, revealing that geometric retention is a structural requirement rather than a design choice.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.05862 [cs.LG] |
| (or arXiv:2605.05862v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.05862 arXiv-issued DOI via DataCite (pending registration) |
From: Yanming Xia [view email]
[v1]
Thu, 7 May 2026 08:31:33 UTC (8,891 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。