Abstract:Bayesian (deep) neural networks (BNN) are often more attractive than the vanilla point-estimate deep learning in various aspects including uncertainty quantification, robustness to noise, resistance to overfitting, and more. The variational inference (VI) is one of the most widely adopted approximate inference methods. Whereas the ELBO-based variational free energy method is a dominant choice in the literature, in this paper we introduce a score-based alternative for BNN variational inference. Score-based VI can address the known issue of mode collapsing in ELBO-based VI. Although several score-based VI methods have been proposed in the community, most are not adequate for large-scale BNNs for various computational and technical reasons. We propose a novel scalable VI method where the learning objective combines the score matching loss and the proximal penalty term in iterations, which helps our method avoid the reparametrized sampling, and allows for noisy unbiased mini-batch scores through stochastic gradients. This in turn makes our method scalable to large-scale neural networks including Vision Transformers. On several benchmarks including visual recognition and time-series forecasting with large-scale deep networks, we empirically show the effectiveness of our approach.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.05873 [cs.LG] |
| (or arXiv:2602.05873v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.05873 arXiv-issued DOI via DataCite |
From: Minyoung Kim [view email]
[v1]
Thu, 5 Feb 2026 16:51:07 UTC (3,578 KB)
[v2]
Thu, 21 May 2026 17:55:14 UTC (3,586 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。