






















Abstract:Learned representations are often invariant to rotational transformations, leaving individual dimensions non-identifiable and interchangeable. We study how Matryoshka Representation Learning (MRL) induces a task-aligned privileged basis distinct from variance-based or regularizer-induced orderings. In the linear setting, we prove that full-prefix MRL recovers the ordered principal directions, and can be computed efficiently using shared statistics. Empirically, we demonstrate that MRL yields consistent per-dimension structure aligned with task signal, where coordinate magnitude reflects informativeness.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.09160 [cs.LG] |
| (or arXiv:2605.09160v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09160 arXiv-issued DOI via DataCite (pending registration) |
From: Arghamitra Talukder [view email]
[v1]
Sat, 9 May 2026 20:44:13 UTC (1,250 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。