Abstract:Single-task RL agents are typically trained under a fixed reward function, which limits their robustness to reward misspecification and their ability to adapt to changing preferences. We introduce Reward-Conditioned Reinforcement Learning (RCRL), an off-policy method that conditions agents on reward parameterizations while collecting experience under a single nominal objective. By recomputing counterfactual rewards from shared replay data, RCRL exposes the agent to multiple reward objectives without additional environment interaction, connecting single-task RL with ideas from multi-objective and multi-task learning. Across single-task, multi-task, and vision-based benchmarks, RCRL improves sample efficiency under the nominal reward parameterization, enables efficient adaptation to new parameterizations, and supports zero-shot behavioral adjustment at deployment. Our results show that RCRL provides a scalable mechanism for learning robust, steerable policies without sacrificing the simplicity of single-task training.
| Comments: | preprint |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2603.05066 [cs.LG] |
| (or arXiv:2603.05066v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.05066 arXiv-issued DOI via DataCite |
From: Michal Nauman [view email]
[v1]
Thu, 5 Mar 2026 11:29:17 UTC (1,535 KB)
[v2]
Sat, 9 May 2026 16:40:23 UTC (1,582 KB)
[v3]
Tue, 19 May 2026 13:19:11 UTC (1,582 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。