























Abstract:Differentially private stochastic gradient descent (DP-SGD) is the gold standard for training machine learning models with formal differential privacy guarantees. Several recent extensions improve its accuracy by introducing correlated noise across training iterations. Matrix factorization mechanisms are a prominent example, but they correlate noise across many iterations and require storing previously added noise vectors, leading to substantial memory overhead in some settings. In this work, we propose a new noise correlation strategy that correlates noise only with the immediately preceding iteration and cancels a controlled portion of it. Our method relies on noise regeneration using a pseudorandom noise generator, eliminating the need to store past noise. As a result, it requires no additional memory beyond standard DP-SGD. We show that the computational overhead is minimal and empirically demonstrate improved accuracy over DP-SGD.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.22334 [cs.LG] |
| (or arXiv:2601.22334v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.22334 arXiv-issued DOI via DataCite |
From: Nikita P. Kalinin [view email]
[v1]
Thu, 29 Jan 2026 21:21:34 UTC (230 KB)
[v2]
Mon, 11 May 2026 20:34:37 UTC (230 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。