Abstract:This paper develops a switching-system interpretation of Q-learning with linear function approximation (LFA) based on the joint spectral radius (JSR). We derive an exact linear switched model for the mean dynamics and relate convergence to stability of the corresponding switched system. The same construction is then used for stochastic linear Q-learning with independent and identically distributed (i.i.d.) observations and with Markovian observations. Although exact JSR computation is difficult in general, the certificate captures products of switching modes and can be less conservative than one-step norm bounds. The framework also yields a JSR-based view of regularized Q-learning with LFA. The resulting analysis connects projected Bellman equations, finite-difference stochastic-policy switching, and switched-system stability in a single parameter-space formulation.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.11021 [cs.LG] |
| (or arXiv:2605.11021v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.11021 arXiv-issued DOI via DataCite (pending registration) |
From: Donghwan Lee [view email]
[v1]
Sun, 10 May 2026 16:21:31 UTC (213 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。