Abstract:Diffusion models have recently emerged as expressive policy representations for online reinforcement learning (RL). However, their iterative generative processes introduce substantial training and inference overhead. To overcome this limitation, we propose to represent policies using MeanFlow models, a class of few-step flow-based generative models, to improve training and inference efficiency over diffusion-based RL approaches. To promote exploration, we optimize MeanFlow policies under the maximum entropy RL framework via soft policy iteration, and address two key challenges specific to MeanFlow policies: action likelihood evaluation and soft policy improvement. Experiments on MuJoCo and DeepMind Control Suite benchmarks demonstrate that our method, Mean Flow Policy Optimization (MFPO), achieves performance comparable to or exceeding current diffusion-based baselines while considerably reducing training and inference time. Our code is available at this https URL.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.14698 [cs.LG] |
| (or arXiv:2604.14698v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.14698 arXiv-issued DOI via DataCite |
From: Xiaoyi Dong [view email]
[v1]
Thu, 16 Apr 2026 06:59:52 UTC (595 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。