Hermes Agent runtime evaluation
HermesBench evaluates complete Hermes configurations: prompt, model/provider, tools, AgentSkills, memory, gateway behavior, delegation, safety, latency, and stability. The current public baseline scores 78.2 across 27 personal-agent recipes with redacted traces you can inspect.
78.2 current public baseline
27 workflow recipes
9 scored suites
Why trust it
Every published result links back to scenario definitions, public score axes, driver closure decisions, deterministic checks, and redacted trace timelines. The site is deliberately clear that this is one early baseline, not a base-model leaderboard.
Site map
With one baseline published, a leaderboard is premature. The site now starts from the content people need to navigate: recipes, profiles, and traces.
Agent-driven quick start
The public user pathway is intentionally simple: copy the prompt to Codex, Claude, or another coding agent. The agent loads the HermesBench skill and drives one scenario recipe first. Full bundle runs are opt-in because they take longer and cost more.
Prompt to copy into Codex or Claude
Use the HermesBench skill and run one default scenario recipe for my current Hermes configuration.
Skill: https://github.com/verkyyi/hermesbench/blob/main/agent-skills/hermesbench/SKILL.md
Follow the skill's "Run Current Hermes Configuration" workflow. Use the Python API default single-recipe path, save artifacts, and summarize the score and main findings. Do not run the full bundle unless I explicitly ask.Alpha feedback
HermesBench needs early feedback on setup friction, scoring surprises, recipe realism, profile evidence, and redaction trust. Star the repo if the benchmark shape is useful; open an issue if one recipe, trace, or score axis feels wrong.
Coverage model
HermesBench starts with one valuable workflow recipe, then lets you opt into broader suites when you need more confidence. The bundled catalog covers everyday personal-agent work: context, calendar, web, reports, communication, location, travel, finance, safety, and power-user integrations.
Personal core Communications Ambient and travel Private sensitive Power-user optional
Scoring philosophy
Outcome reached Evidence / truthfulness Runtime / scope safety Responsiveness Task fulfillment Communication quality
HermesBench is reliability-first, but not capability-blind. A good configuration should do useful work, tell the truth about what it knows, avoid unsafe side effects, stay stable, respond promptly, and communicate clearly. Lopsided scores are penalized because a personal agent that is capable but unsafe, safe but unhelpful, or correct but unusably slow is not actually good.
Detailed formulas and implementation mechanics live in the methodology document; the website keeps the scoring model readable for users and LLM agents.
Use and contribute
HermesBench is useful as a quick benchmark, but it is also a way to publish what worked. Share a redacted profile/config package when a setup improves a recipe, or submit a generic recipe when an important personal-agent use case is missing.
Profile submission prompt
Use the HermesBench skill to prepare my current Hermes profile/config as a public profile submission.
Skill: https://github.com/verkyyi/hermesbench/blob/main/agent-skills/hermesbench/SKILL.md
Run one representative recipe first, package the redacted profile snapshot and score evidence, and tell me what must be reviewed before opening a pull request.Recipe submission prompt
Use the HermesBench skill to propose a new generic personal-agent recipe for HermesBench.
Skill: https://github.com/verkyyi/hermesbench/blob/main/agent-skills/hermesbench/SKILL.md
Make the use case privacy-safe, driver/target agnostic, fixture-backed where possible, and include deterministic checks before preparing a pull request.此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。