Authors
- Xiangyu Liu Nanjing University
- Haodi Lei Nanjing University
- Yi Liu Nanjing University
- Yang Liu Nanjing University
- Wei Hu Nanjing University
DOI:
https://doi.org/10.1609/aaai.v40i1.37044Abstract
Sparse Autoencoder (SAE) has emerged as a powerful tool for mechanistic interpretability of large language models. Recent works apply SAE to protein language models (PLMs), aiming to extract and analyze biologically meaningful features from their latent spaces. However, SAE suffers from semantic entanglement, where individual neurons often mix multiple nonlinear concepts, making it difficult to reliably interpret or manipulate model behaviors. In this paper, we propose a semantically-guided SAE, called ProtSAE. Unlike existing SAE which requires annotation datasets to filter and interpret activations, we guide semantic disentanglement during training using both annotation datasets and domain knowledge to mitigate the effects of entangled attributes. We design interpretability experiments showing that ProtSAE learns more biologically relevant and interpretable hidden features compared to previous methods. Performance analyses further demonstrate that ProtSAE maintains high reconstruction fidelity while achieving better results in interpretable probing. We also show the potential of ProtSAE in steering PLMs for downstream generation tasks.
How to Cite
Liu, X., Lei, H., Liu, Y., Liu, Y., & Hu, W. (2026). ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders. Proceedings of the AAAI Conference on Artificial Intelligence, 40(1), 773–781. https://doi.org/10.1609/aaai.v40i1.37044
Issue
Section
AAAI Technical Track on Application Domains I