Abstract:Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift regularization provides a natural remedy to disentangle task vectors; however, existing approaches typically require external task data, conflicting with modularity and data availability constraints (e.g., privacy requirements). We propose a dataless approach by framing regularization against representation drift as a curvature matrix approximation problem. This allows us to leverage well-established techniques; in particular, we adopt Kronecker-Factored Approximate Curvature and obtain a practical regularizer that achieves state-of-the-art results in task addition and negation. Our method has constant complexity in the number of tasks and promotes robustness to task vector rescaling, eliminating the need for held-out tuning.
| Comments: | Accepted to ICLR 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.17385 [cs.AI] |
| (or arXiv:2602.17385v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.17385 arXiv-issued DOI via DataCite |
From: Angelo Porrello [view email]
[v1]
Thu, 19 Feb 2026 14:10:45 UTC (11,359 KB)
[v2]
Mon, 23 Feb 2026 17:36:15 UTC (11,359 KB)
[v3]
Thu, 21 May 2026 10:11:44 UTC (10,891 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。