




















Abstract:Finding Nash equilibria in two-player zero-sum imperfect-information games remains a central challenge in multi-agent reinforcement learning. Recent multi-round regularization methods offer a promising direction, yet existing approaches either require full enumeration of the game tree or rely on non-policy-gradient inner solvers that underperform in practice, leaving a scalable policy-gradient-based solution open. In this paper, we propose a novel multi-round regularization procedure and show that it guarantees strictly monotonic reduction in Bregman divergence to Nash equilibria and eventual convergence to one in two-player zero-sum extensive-form games. Guided by this framework, we develop a practical algorithm, Nash Policy Gradient (NashPG), which places the regularization directly in the policy optimization objective and is implemented using standard policy gradient methods. Empirically, NashPG achieves comparable or lower exploitability than prior model-free methods on classic benchmark games and scales to large domains such as Battleship and No-Limit Texas Hold'em, where it attains higher average payoff in head-to-head play.
| Subjects: | Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT) |
| Cite as: | arXiv:2510.18183 [cs.LG] |
| (or arXiv:2510.18183v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.18183 arXiv-issued DOI via DataCite |
From: Eason Yu [view email]
[v1]
Tue, 21 Oct 2025 00:14:45 UTC (239 KB)
[v2]
Thu, 30 Apr 2026 16:28:49 UTC (212 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。