Abstract:Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.
| Comments: | Alexandre Polo and Chanwoo Chun contributed equally to this work |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.20338 [cs.LG] |
| (or arXiv:2602.20338v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.20338 arXiv-issued DOI via DataCite |
From: Chanwoo Chun [view email]
[v1]
Mon, 23 Feb 2026 20:36:17 UTC (8,207 KB)
[v2]
Fri, 8 May 2026 02:14:05 UTC (8,476 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。