Abstract:For reinforcement learning in the real world online exploration is expensive A common practice in robotic reinforcement learning is to incorporate additional data to improve sample efficiency Expert demonstration data is often crucial for solving hard exploration tasks with sparse rewards While prior data is used to augment experience and pretrain models we show that the design of existing algorithms fails to achieve the sample efficiency that is possible in this setting due to a failure to use pretrained policies effectively We propose XQCfD which extends the sample-efficient XQC actor-critic to learn from demonstrations using augmented replay buffers pretrained policies and stationary policy architectures designed to avoid rapidly unlearning the strong initial policy like prior works We show our stationary network architecture enables policy improvement out-of-distribution better than standard network architectures due to its higher entropy predictions XQCfD achieves state of the art performance across a range of complex manipulation tasks with sparse rewards from the popular Adroit Robomimic and MimicGen benchmarks -- notably with a low update-to-data ratio and no ensemble networks
| Comments: | 22 pages, 10 figures, 2 tables |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.10734 [cs.LG] |
| (or arXiv:2605.10734v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.10734 arXiv-issued DOI via DataCite (pending registration) |
From: Joe Watson [view email]
[v1]
Mon, 11 May 2026 15:38:25 UTC (1,808 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。