Interpretability via sparse autoencoders can mask extreme fragility in the very neurons we treat as safety handles. A single clamped unit may look like a clean lever, yet the model can reroute around it without our notice. The illusion of control grows as practitioners equate “human‑readable” with “trustworthy.”
Sparse autoencoders have become the de‑facto tool for dissecting residual‑stream activations, and many recent defenses assume that clamping a identified unsafe feature will reliably suppress the corresponding misbehavior. This assumption underpins latent‑space steering, refusal‑steering, and unlearning pipelines that intervene directly on SAE latents.
Stable SAE features concentrate the predictive power, while unstable features barely move the needle. The authors report a “functional asymmetry of stable and unstable features. Stable features carry most of the reconstruction‑ and prediction‑relevant signal, whereas unstable features have weak marginal impact and are dominated by low‑frequency surface‑form triggers” [1]. Moreover, unstable features fire on average only 0.18 % of tokens versus 0.44 % for stable ones, highlighting their sporadic influence.
Despite their individual non‑reproducibility, unstable features live in a reproducible low‑rank subspace. “Decoder‑space analysis shows that unstable features are individually non‑reproducible but collectively span reproducible lower‑rank subspaces” [1], suggesting that seed‑dependent basis choices shuffle the same underlying geometry rather than generate pure noise.
Even a hard‑wired clamp on a harmful feature can be undone on almost every test case. In safety‑critical refusal‑steering, the authors achieve a 95.8 % recovery rate on valid samples while keeping defended‑feature relative drift to 0.131, “substantially below suffix‑based baselines” [2].
The SAE reconstruction residual, not the clamped concepts, carries the recovered unsafe behavior. “Figure 6 identifies the SAE reconstruction residual as the dominant carrier. Residual replay nearly matches full recovery, while clamped‑feature replay fails and non‑clamped SAE‑feature replay remains weak” [2].
These results expose a gap between feature‑level control and behavioral completeness: “SAE features can support causal intervention, but controlling them does not guarantee control over the underlying behavior” [2]. The interventions block a visible route but leave the underlying functionality intact.
The recovery experiments target a narrow set of prompts and keep the clamp active throughout optimization, so it remains unclear how the phenomenon scales to open‑ended generation or to interventions that are applied intermittently. Likewise, the seed‑stability analysis focuses on reconstruction statistics rather than downstream safety metrics, leaving open whether the identified unstable subspaces actually matter for real‑world harms.
Safety pipelines that treat SAE latents as definitive intervention points should be revised to monitor the residual stream as well. Deployments that rely solely on clamped neurons risk a false sense of security, because the model can re‑express the same undesirable behavior through the unexplained residual.
If every interpretable neuron can be sidestepped, what concrete metric will certify that a model is truly safe?