
























Abstract:Large Language Models (LLMs) frequently hallucinate, limiting their reliability in critical applications. Conformal Prediction (CP) addresses this by calibrating error rates on held-out data to provide statistically valid confidence guarantees. Recent work extends CP to LLM factuality to filter out risky claims, ensuring that hallucination rates remain below a user-specified level (e.g., 10%). While prior methods treat claims independently, Coherent Factuality extends to multi-step reasoning by representing outputs as dependency graphs and jointly validating claims with their logical ancestors. A key limitation is that Coherent Factuality is not differentiable, requiring hand-crafted scorers that at high reliability levels remove nearly 60% of true claims. We introduce Differentiable Coherent Factuality (DCF), a fully differentiable relaxation that enables learning improved scorers while provably recovering the original algorithm's guarantees. Experiments on two benchmark reasoning datasets demonstrate DCF achieves up to 141% improvement in claim retention while maintaining reliability guarantees, representing a significant step towards reliable conformal LLM systems.
| Comments: | Submitted ICML |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.20098 [cs.LG] |
| (or arXiv:2604.20098v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.20098 arXiv-issued DOI via DataCite (pending registration) |
From: Nathan Hittesdorf [view email]
[v1]
Wed, 22 Apr 2026 01:35:31 UTC (600 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。