
























Abstract:Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection of AI safety, moral philosophy, and cognitive science. We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments. By decoupling task-solving from moral evaluation and introducing a novel Morality Metric, MoralityGym allows the integration of insights from psychology and philosophy into the evaluation of norm-sensitive reasoning. Baseline results with Safe RL methods reveal key limitations, underscoring the need for more principled approaches to ethical decision-making. This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.
| Comments: | Accepted at AAMAS 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.13372 [cs.AI] |
| (or arXiv:2602.13372v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.13372 arXiv-issued DOI via DataCite |
|
| Journal reference: | Proc of the 25th International Conference on Autonomous Agents and Multiagent Systems AAMAS 2026, Paphos, Cyprus, May 25 to 29, 2026, IFAAMAS |
| Related DOI: | https://doi.org/10.65109/SAKL6648
DOI(s) linking to related resources |
From: Simon Rosen Mr. [view email]
[v1]
Fri, 13 Feb 2026 15:40:32 UTC (452 KB)
[v2]
Thu, 21 May 2026 13:32:14 UTC (453 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。