Abstract:Sparse goal-conditioned reinforcement learning can produce policies whose failures are hidden by aggregate success rates. We analyze trained goal-conditioned value policies through the deterministic functional graphs induced by greedy evaluation: for each goal, every state maps to a single successor, decomposing behavior into attractors and basins. This reveals a local-to-global structure in learned policies. We define local goal support (LGS), a one-step statistic measuring the fraction of valid neighboring states whose greedy successor is the goal. In deterministic sparse GridWorlds, zero LGS exactly precludes goal entry from non-goal starts. Empirically, weak LGS is a strong diagnostic of goal-level failure across update rules, curricula, larger grids, and bottleneck geometries: the fixed rule LGS <= 0.5 identifies low-success goals with precision 0.921, recall 0.929, and F1 0.925 in the main 8x8 TD setting, with similar performance across variants. However, local support is not sufficient for global success: some supported goals still fail because distant states are captured by competing attractors or fragmented basin structure. We therefore introduce a compact post-hoc taxonomy of policy-induced graphs -- goal-dominant, competitor-dominated, partial/contested, and fragmented -- to characterize residual failure modes beyond local support. These results show that sparse GCRL failures can be understood as structured policy-induced dynamics, and that local one-step policy structure provides a cheap post-training diagnostic for goal-level failure.
| Comments: | 9 pages main, 21 pages appendx, 2 figures in main. 8 figures in appendix, Submitted to a conference |
| Subjects: | Machine Learning (cs.LG) |
| MSC classes: | 68T05, 37M05, 05C82 |
| Cite as: | arXiv:2605.09335 [cs.LG] |
| (or arXiv:2605.09335v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09335 arXiv-issued DOI via DataCite (pending registration) |
From: Shailey Dash Dr [view email]
[v1]
Sun, 10 May 2026 05:16:51 UTC (325 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。