Abstract:Offline Reinforcement Learning (RL) is an emerging field of RL in which policies are learned solely from demonstrations. Within offline RL, some environments involve balancing multiple objectives, but existing multi-objective offline RL algorithms do not provide an efficient way to find a fair compromise. FairDICE (see arXiv:2506.08062v2) seeks to fill this gap by adapting OptiDICE (an offline RL algorithm) to automatically learn weights for multiple objectives to e.g. incentivise fairness among objectives. As this would be a valuable contribution, this replication study examines the replicability of claims made regarding FairDICE. We find that many theoretical claims hold, but an error in the code reduces FairDICE to standard behaviour cloning in continuous environments, and many important hyperparameters were originally underspecified. After rectifying this, we show in experiments extending the original paper that FairDICE can scale to complex environments and high-dimensional rewards, though it can be reliant on (online) hyperparameter tuning. We conclude that FairDICE is a theoretically interesting method, but the experimental justification requires significant revision.
| Comments: | 12 pages, 8 figures in main text. Code at this https URL. Reviewed at this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2603.03454 [cs.LG] |
| (or arXiv:2603.03454v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.03454 arXiv-issued DOI via DataCite |
|
| Journal reference: | Published 05/2026 in Transactions on Machine Learning Research |
From: Peter Adema [view email]
[v1]
Tue, 3 Mar 2026 19:12:06 UTC (9,124 KB)
[v2]
Thu, 21 May 2026 12:57:18 UTC (9,153 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。