























Abstract:Reinforcement learning (RL) is a type of artificial intelligence for making optimal choices. In healthcare, researchers generally use offline RL (ORL), where models are trained and evaluated from retrospective observational data. To accommodate inherently irregular clinical records, researchers often resample the data into uniform time intervals before training (known as binning). However, discretised data presents the model with a fictional representation of clinical scenarios, especially where unpredictable decision timings are common. As these models lack robust trial evidence, we chose to explore the effects of this further by conducting an in silico clinical trial using 30 virtual patients with type 1 diabetes from the FDA-approved UVA/Padova simulator. The simulator was modified to include stochastic intervals between decisions and used to generate a training dataset for offline RL. We trained three ORL algorithms on both the unprocessed dataset and equivalent datasets resampled at 10-minute, 2-hour, and 4-hour intervals. When deployed back into the simulated environment, temporal resampling was found to reduce model performance by up to 60% relative to unprocessed data, with 4-hour binning causing all agents to perform worse than the dataset's baseline. Retrospective evaluation on resampled data actively obscured this effect, predicting 1.5-3x better returns than agents achieved in practice. We recommend that future research in this area prioritises datasets with natural clinical timings between decisions, which may be a necessary step before these models can be safely deployed into patient care.
| Comments: | 12 pages, 6 figures, 3 tables. v3 updates with lit rev table |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.06603 [cs.LG] |
| (or arXiv:2602.06603v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.06603 arXiv-issued DOI via DataCite |
From: Thomas Frost [view email]
[v1]
Fri, 6 Feb 2026 11:02:06 UTC (11,981 KB)
[v2]
Tue, 10 Feb 2026 09:51:38 UTC (11,980 KB)
[v3]
Wed, 29 Apr 2026 10:13:20 UTC (3,018 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。