
























Abstract:Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty because they apply a Bayesian treatment only to the final layer, ignoring uncertainty induced by earlier layers. We propose a method that improves BLLs by leveraging a projection of Neural Tangent Kernel (NTK) features onto the space spanned by the last-layer features. This enables posterior inference that accounts for variability of the full network while retaining the low computational cost of inference of a standard BLL. We show that our method yields posterior variances that are provably greater or equal to those of a standard BLL, correcting its tendency to underestimate epistemic uncertainty. To further reduce computational cost, we introduce a uniform subsampling scheme for estimating the projection matrix and for posterior inference. We derive approximation bounds for both types of subsampling. Empirical evaluations on UCI regression, contextual bandits, image classification, and out-of-distribution detection tasks in image and tabular datasets, demonstrate improved calibration and uncertainty estimates compared to standard BLLs and competitive baselines, while reducing computational cost.
| Comments: | Appearing in the Proceedings of the 43rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.01279 [cs.LG] |
| (or arXiv:2602.01279v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.01279 arXiv-issued DOI via DataCite |
From: Sergio Calvo-Ordoñez [view email]
[v1]
Sun, 1 Feb 2026 15:24:20 UTC (511 KB)
[v2]
Wed, 20 May 2026 23:01:12 UTC (533 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。