




























Abstract:Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic programming, this approach has received relatively less attention, mainly because it is often less efficient in practice and can be more difficult to extend to model-free settings such as reinforcement learning. Nonetheless, Bellman residual minimization has several advantages that make it worth investigating, such as more stable convergence with function approximation for value functions. While Bellman residual methods for policy evaluation have been widely studied, methods for policy optimization (control tasks) have been scarcely explored. In this paper, we establish foundational results for the control Bellman residual minimization for policy optimization.
| Subjects: | Machine Learning (cs.LG); Systems and Control (eess.SY) |
| Cite as: | arXiv:2601.18840 [cs.LG] |
| (or arXiv:2601.18840v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.18840 arXiv-issued DOI via DataCite |
From: Donghwan Lee [view email]
[v1]
Mon, 26 Jan 2026 10:58:27 UTC (1,826 KB)
[v2]
Wed, 11 Feb 2026 09:33:08 UTC (1,826 KB)
[v3]
Sun, 26 Apr 2026 23:40:47 UTC (2,151 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。