Abstract:Reinforcement learning (RL) in healthcare has had mixed results, with reward sparsity, unreliable off-policy evaluation, and deployment-simulation gap as recurring failure modes. We argue that chronic disease management is structurally a more tractable RL setting than the acute-care problems the field has primarily studied, but only if the problem is formalized to exploit chronic care's properties. We propose such a formalization. The agent's objective is to compress time-to-control (TTC) under a tiered reward calibrated to the CMS ACCESS Model. Two quantities from our companion preference-learning paper [Singh et al. 2026] enter as load-bearing structural elements: the execution intensity \epsilon bounds action availability under a constrained Markov Decision Process, and the clinician capability \kappa weights offline-data transitions during RL training. Together they couple preference learning and RL into a two-loop architecture. We present simulation results on synthetic state machines for hypertension and type 2 diabetes. Capability-weighted offline RL outperforms uniform-weighted offline RL and the behavior policy by 15 percentage points on T2D TTC; the uniform-weighted formulation (the standard in existing healthcare RL) underperforms even the heterogeneous behavior policy. \Epsilon-aware policies generalize across deployment regimes while \epsilon-naive policies do not.
| Comments: | 26 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.09818 [cs.LG] |
| (or arXiv:2605.09818v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09818 arXiv-issued DOI via DataCite (pending registration) |
From: Prabhjot Singh [view email]
[v1]
Sun, 10 May 2026 23:43:39 UTC (55 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。