Abstract:Accurate uncertainty quantification remains a key challenge for standard LLMs, prompting the adoption of Bayesian and ensemble-based methods. However, such methods typically necessitate computationally expensive sampling, involving multiple forward passes to effectively estimate predictive uncertainty.
In this paper, we introduce an approach enabling uncertainty estimation in LLMs without incurring the heavy inference latency typically associated with sampling methods. Specifically, we distill uncertainty-aware teachers - originally requiring multiple forward passes - into single-pass students, fine-tuned using LoRA. We compare two distinct distillation strategies: one in which the student employs traditional softmax-based outputs, and another in which the student leverages Dirichlet-distributed outputs to explicitly model epistemic uncertainty via evidential learning.
Empirical evaluation on classification tasks demonstrate that such students can achieve comparable predictive and uncertainty quantification performance relative to their teachers, while requiring only a single forward pass.
From: Tomasz Kuśmierczyk [view email]
[v1]
Thu, 24 Jul 2025 12:46:40 UTC (349 KB)
[v2]
Mon, 6 Jul 2026 15:29:53 UTC (371 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。