Abstract:Finite-width fully connected neural networks with Gaussian-initialized weights deviate from their infinite-width Gaussian limit, exhibiting non-vanishing higher-order cumulants. We approximate these deviations, for a neural network evaluated in a finite number of inputs, using multidimensional Edgeworth expansions of arbitrary order $4m-1$, with $m\in\mathbb{N}$. Assuming that the corresponding Gaussian limit has an invertible covariance matrix and that the activation function is polynomially bounded, we establish a bound of order $n^{-m}$ on the total variation distance between the law of the true network output and its Edgeworth approximation, with matching lower bounds. As an application, we quantify the error in Bayesian posterior distributions when the prior is replaced by its Edgeworth expansion. Our results are more general and also apply to sequences of conditionally Gaussian vectors converging to a Gaussian vector with invertible covariance.
| Comments: | 34 pages, 2 figures |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR) |
| MSC classes: | 60E10 (Primary) 60F05, 60G15, 60G60, 68T07, 62C10 (Secondary) |
| ACM classes: | G.3; I.2 |
| Cite as: | arXiv:2605.24072 [stat.ML] |
| (or arXiv:2605.24072v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2605.24072 arXiv-issued DOI via DataCite (pending registration) |
From: Lucia Celli [view email]
[v1]
Fri, 22 May 2026 12:45:17 UTC (946 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。