Abstract:While researchers are finding concepts represented as linear directions in language models, a bag of linear directions fails to capture relational structure. To better understand this dichotomy, we study a model with known linear representations, but trained in a highly structured domain -- the board game Othello. While the model's internal board-state representation is linearly decodable, we find additional structure in the form of tensor product representations (TPRs). We train TPR probes to recover shared structure amongst the linear probes, yielding a factorization into square-embeddings, color-embeddings, and a binding matrix that composes them to construct the model's board-state representation. We find geometric signatures within the weights of our TPR probe that align with the structure of the board, but perhaps more importantly, that the linear probes can be recovered directly from the parameters of our TPR probe. Our findings suggest that directional representations may be projections of more structured underlying representations.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.09967 [cs.LG] |
| (or arXiv:2605.09967v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09967 arXiv-issued DOI via DataCite (pending registration) |
From: Andrew Lee [view email]
[v1]
Mon, 11 May 2026 04:18:43 UTC (772 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。