Abstract:Recurrent networks can contain substantial functional redundancy in
weight space: changing a recurrent matrix may leave the input-output
rollout nearly unchanged on a task distribution, while similar-scale
changes can destroy the same behavior. We study this redundancy in
one-layer tanh RNNs using ordered real Schur coordinates. The Schur
form separates spectral blocks from directed nonnormal couplings,
giving a diagnostic basis for structured ablations that keep the input
and readout maps fixed. In a fixed-length copy task, selected
nonnormal Schur couplings can be removed with little loss in some
trained solutions, whereas other couplings are necessary for accurate
autonomous replay. Across flip-flop, sine generation, and
context-dependent integration, the loss-preserving ablation profile
varies across tasks and trained solutions. These results identify
candidate approximate functional invariances, not universal symmetries
of recurrent weight space. Schur-coordinate ablations provide a
practical diagnostic for which structured perturbations preserve a
trained recurrent solution and which ones disrupt its computation.
From: Simon Dräger [view email]
[v1]
Tue, 16 Jun 2026 20:04:07 UTC (471 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。