


























Abstract:The remarkable success of the Adam in training neural networks has naturally led to the widespread use of its descent-ascent counterpart, Adam-DA, for solving zero-sum games. Despite its popularity in practice, a rigorous theoretical understanding of Adam-DA still lags behind. In this paper, we derive ordinary differential equations (ODEs) that serve as continuous-time limits of the Adam-DA. These ODEs closely approximate the discrete-time dynamics of Adam-DA, providing a tractable analytical framework for understanding its behavior in zero-sum games. Using this ODE approach, we investigate two fundamental aspects of Adam-DA: local convergence and implicit gradient regularization. Our analysis reveals that the roles of the first- and second-order momentum parameters in zero-sum games are exactly the opposite of their well-documented effects in minimization problems. We validate these predictions through GAN experiments across multiple architectures and datasets, demonstrating the practical implications of this reversed momentum effect.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.19392 [cs.LG] |
| (or arXiv:2605.19392v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19392 arXiv-issued DOI via DataCite (pending registration) |
From: Weiming Ou [view email]
[v1]
Tue, 19 May 2026 05:38:47 UTC (21,154 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。