Abstract:Scaling inference methods such as Markov chain Monte Carlo to high-dimensional models remains a central challenge in Bayesian deep learning. A promising recent proposal, microcanonical Langevin Monte Carlo, has shown state-of-the-art performance across a wide range of problems. However, its reliance on full-dataset gradients makes it prohibitively expensive for large-scale problems. This paper addresses a fundamental question: Can microcanonical dynamics effectively leverage mini-batch gradient noise? We provide the first systematic study of this problem, establishing a novel continuous-time theoretical analysis of stochastic-gradient microcanonical dynamics. We reveal two critical failure modes: a theoretically derived bias due to anisotropic gradient noise and numerical instabilities in complex high-dimensional posteriors. To tackle these issues, we propose a principled gradient noise preconditioning scheme shown to significantly reduce this bias and develop a novel, energy-variance-based adaptive tuner that automates step size selection and dynamically informs numerical guardrails. The resulting algorithm is a robust and scalable microcanonical Monte Carlo sampler that achieves state-of-the-art performance on challenging high-dimensional inference tasks like Bayesian neural networks. Combined with recent ensemble techniques, our work unlocks a new class of stochastic microcanonical Langevin ensemble (SMILE) samplers for large-scale Bayesian inference.
| Comments: | In Proceedings of the 43rd International Conference on Machine Learning |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.06500 [cs.LG] |
| (or arXiv:2602.06500v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.06500 arXiv-issued DOI via DataCite |
From: Emanuel Sommer [view email]
[v1]
Fri, 6 Feb 2026 08:52:19 UTC (2,738 KB)
[v2]
Wed, 20 May 2026 05:41:59 UTC (2,751 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。