

















Abstract:Deployed reinforcement learning agents often face safety requirements that are specified only after training, such as new hazard maps, revised risk thresholds, or behavioral alignment constraints. We study zero-update deployment-time adaptation, where a fixed library of risk-neutral source policies is reused under a newly specified reward-risk tradeoff. We propose TRAM (Test-Time Risk Adaptation via Mixture of Agents), a source-scored composition rule that evaluates each source policy under the target reward and an occupancy-based deployment risk, then selects actions using risk-adjusted source scores. Unlike training-time risk-sensitive methods tied to a fixed surrogate such as return variance, TRAM supports spatial barrier exposure, divergence from a reference behavior, and local volatility risks specified at test time. We explicitly characterize TRAM as a surrogate method: it does not solve the full occupancy-control problem of the stitched policy, but admits a measurable source-hull mismatch term connecting source-scored risk to realized risk. Experiments in gridworlds, MuJoCo Reacher, Safety-Gymnasium, and an LLM alignment setting show that TRAM reduces deployment risk while preserving reward, without requiring any parameter updates at test time.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2408.08812 [cs.LG] |
| (or arXiv:2408.08812v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2408.08812 arXiv-issued DOI via DataCite |
From: Mohamad Chehade [view email]
[v1]
Fri, 16 Aug 2024 15:47:08 UTC (659 KB)
[v2]
Wed, 20 May 2026 00:39:52 UTC (9,872 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。