

























Abstract:Large Language Models (LLMs) can enhance reasoning capabilities through test-time scaling by generating multiple traces. However, the combination of lengthy reasoning traces with multiple sampling introduces substantial computation and high end-to-end latency. Prior work on accelerating this process has relied on similarity-based or confidence-based pruning, but these signals do not reliably indicate trace quality. To address these limitations, we propose STEP: Step-level Trace Evaluation and Pruning, a novel pruning framework that evaluates reasoning steps using hidden states and dynamically prunes unpromising traces during generation. We train a lightweight step scorer to estimate trace quality, and design a GPU memory-aware pruning strategy that triggers pruning as the GPU memory is saturated by KV cache to reduce end-to-end latency. Experiments across challenging reasoning benchmarks demonstrate that STEP reduces end-to-end inference latency by 45%-70% on average compared to self-consistency while also improving reasoning accuracy. Our code is released at: this https URL
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.09093 [cs.LG] |
| (or arXiv:2601.09093v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.09093 arXiv-issued DOI via DataCite |
From: Beichen Huang [view email]
[v1]
Wed, 14 Jan 2026 02:54:55 UTC (398 KB)
[v2]
Tue, 28 Apr 2026 04:40:33 UTC (413 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。