Abstract:State-Space Models (SSMs) excel at capturing long-range dependencies with structured recurrence, making them well-suited for sequence modeling. However, their evolving internal states pose challenges in adapting them under Continual Learning (CL). This is particularly difficult in exemplar-free settings, where the absence of prior data leaves updates to the dynamic SSM states unconstrained, resulting in catastrophic forgetting. To address this, we propose Inf-SSM, a novel and simple geometry-aware regularization method that utilizes the geometry of the infinite-dimensional Grassmannian to constrain state evolution during CL. Unlike classical continual learning methods that constrain weight updates, Inf-SSM regularizes the infinite-horizon evolution of SSMs encoded in their extended observability subspace. We show that enforcing this regularization requires solving a matrix equation known as the Sylvester equation, which typically incurs $\mathcal{O}(n^3)$ complexity. We develop a $\mathcal{O}(n^2)$ solution by exploiting the structure and properties of SSMs. This leads to an efficient regularization mechanism that can be seamlessly integrated into existing CL methods. Comprehensive experiments on challenging benchmarks, including ImageNet-R and Caltech-256, demonstrate a significant reduction in forgetting while improving accuracy across sequential tasks.
| Comments: | Accepted at CVPR 2026 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2505.18604 [cs.LG] |
| (or arXiv:2505.18604v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2505.18604 arXiv-issued DOI via DataCite |
From: Isaac Ning Lee [view email]
[v1]
Sat, 24 May 2025 08:59:13 UTC (276 KB)
[v2]
Sun, 22 Mar 2026 07:20:48 UTC (1,170 KB)
[v3]
Wed, 13 May 2026 05:15:53 UTC (1,170 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。