

















Abstract:With the advent of Large Language Models (LLMs), general-purpose agents have seen fundamental advancements. However, evaluating these agents presents unique challenges that distinguish them from static QA benchmarks. We observe that current agent benchmarks are heavily confounded by extraneous factors, including system prompts, toolset configurations, and environmental dynamics. Existing evaluations often rely on fragmented, researcher-specific frameworks where the prompt engineering for reasoning and tool usage varies significantly, making it difficult to attribute performance gains to the model itself. Additionally, the lack of standardized environmental data leads to untraceable errors and non-reproducible results. This lack of standardization introduces substantial unfairness and opacity into the field. We propose that a unified evaluation framework is essential for the rigorous advancement of agent evaluation. To this end, we introduce a proposal aimed at standardizing agent evaluation.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.03238 [cs.AI] |
| (or arXiv:2602.03238v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.03238 arXiv-issued DOI via DataCite |
From: Pengyu Zhu [view email]
[v1]
Tue, 3 Feb 2026 08:18:37 UTC (2,092 KB)
[v2]
Tue, 26 May 2026 10:32:10 UTC (7,467 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。