

















Abstract:Sparse autoencoders (SAEs) are a mechanistic interpretability technique that have been used to provide insight into learned concepts within large protein language models. Here, we employ TopK and Ordered SAEs to investigate autoregressive antibody language models, and steer their generation. We show that TopK SAEs can reveal biologically meaningful latent features, but high feature-concept correlation does not guarantee causal control over generation. In contrast, Ordered SAEs impose a hierarchical structure that reliably identifies steerable features, but at the expense of more complex and less interpretable activation patterns. These findings advance the mechanistic interpretability of domain-specific protein language models and suggest that, while TopK SAEs suffice for mapping latent features to concepts, Ordered SAEs are preferable when precise generative steering is required.
| Comments: | v3: 15 pages; corrected author list and affiliations in the main text; minor text changes; updated steering results following minor code changes; conclusions and findings remain unchanged; included link to data and code in the Data Availability section |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM) |
| Cite as: | arXiv:2512.05794 [cs.LG] |
| (or arXiv:2512.05794v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.05794 arXiv-issued DOI via DataCite |
From: Rebonto Haque [view email]
[v1]
Fri, 5 Dec 2025 15:18:50 UTC (5,593 KB)
[v2]
Fri, 24 Apr 2026 17:16:11 UTC (34,398 KB)
[v3]
Tue, 26 May 2026 15:50:47 UTC (35,712 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。