Abstract:Conventional federated learning assumes that greater learner participation improves training performance, by leveraging abundant, independently generated local data. However, in federated reinforcement learning (FRL) for unmanned aerial vehicle (UAV) teams in hazardous environments where experience generation is severely constrained by safety considerations, energy limitations, and mission duration, this assumption may break. This work introduces Experience-Constrained Hierarchical Federated Reinforcement Learning (EC-HFRL), a framework in which clusters act as federated learning agents, while multiple intra-cluster learners represent parallel learning resources that reuse a shared experience pool. We show that increasing participation does not necessarily improve learning performance. Instead, learning performance is strongly associated with experience reuse strategy and the dominance of key analytically identified gradient transition experiences within a cluster. In particular, minibatch size primarily determines effective replay exposure, while higher intra-cluster participation increases reuse level. Empirical results demonstrate that the performance regimes are strongly associated with the structure of the learning signal, rather than federated aggregation effects, clarifying the limited and secondary role of learner participation in experience-constrained FRL.
| Comments: | Accepted by the International Joint Conference on Neural Networks (IJCNN 2026), part of WCCI 2026 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.02165 [cs.LG] |
| (or arXiv:2605.02165v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.02165 arXiv-issued DOI via DataCite (pending registration) |
From: Qinwei Huang [view email]
[v1]
Mon, 4 May 2026 02:46:09 UTC (1,973 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。