Abstract:Large reasoning models (LRMs) often generate long, seemingly coherent reasoning traces yet still produce incorrect answers, making hallucination detection challenging. Although trajectories contain useful signals, directly using trace text or vanilla hidden states for detection is brittle: traces vary in form and detectors can overfit to superficial patterns rather than answer validity. We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability. ARS generates counterfactual answers through small latent interventions, specifically, perturbing the trace-boundary embedding, and labels each perturbation by whether the resulting answer agrees with the original. It then learns representations that bring answer-agreeing states together and separate answer-disagreeing ones, exposing latent instability indicative of hallucination risk. The shaped embeddings are plug-and-play with existing embedding-based detectors and require no human annotations during training. Experiments demonstrate that ARS consistently improves detection and achieves substantial gains over strong baselines. Code is available at: this https URL.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.17467 [cs.LG] |
| (or arXiv:2601.17467v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.17467 arXiv-issued DOI via DataCite |
From: Jianxiong Zhang [view email]
[v1]
Sat, 24 Jan 2026 13:47:51 UTC (4,074 KB)
[v2]
Mon, 4 May 2026 13:20:43 UTC (4,088 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。