

























Abstract:Reinforcement learning with verifiable rewards (RLVR) has significantly boosted the reasoning capability of language models (LMs). However, existing RLVR approaches train LMs based on their own on-policy responses and are constrained by the initial capability of LMs, thus prone to exploration stagnation, in which LMs fail to solve more training problems and cannot further learn from the training data. Some approaches try to address this by leveraging off-policy solutions to training problems, but rely on external expert guidance that is limited in availability and scalability. In this work, we propose LTE (Learning to reason from Trial and Error), an approach that hints LMs with their previously self-made mistakes, not requiring any external expert guidance. Experiments validate the effectiveness of LTE, which outperforms the normal group relative policy optimization (GRPO) by 5.02 in Pass@1 and 9.96 in Pass@k on average across six mathematical reasoning benchmarks for Qwen3-8B-Base and even performs better than methods that require external guidance. Further analysis confirms that LTE successfully mitigates exploration stagnation and enhances both exploitation and exploration during training. Our code is available at this https URL.
| Comments: | Accepted to ACL 2026 (main conference) |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.26109 [cs.LG] |
| (or arXiv:2510.26109v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.26109 arXiv-issued DOI via DataCite |
From: Chenming Tang [view email]
[v1]
Thu, 30 Oct 2025 03:36:19 UTC (221 KB)
[v2]
Mon, 5 Jan 2026 12:47:18 UTC (336 KB)
[v3]
Tue, 6 Jan 2026 11:33:27 UTC (336 KB)
[v4]
Thu, 16 Apr 2026 04:58:10 UTC (340 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。