Challenge Entry for the Hermes Agent Challenge

🚀 What Problem Does This Solve?

AI agent deployments often suffer from two critical blind spots:

1. Cost visibility — you discover a $500 OpenAI bill at the end of the month with no clue which cron jobs or sessions caused it
2. Budget control — runaway loops or expensive model choices can drain your account before you notice

hermes-telemetry solves both by giving you real-time observability and automatic budget enforcement for Hermes Agent.

🎯 Why This Plugin Matters

Every production AI system needs observability and cost control. This isn't just a nice-to-have — it's essential infrastructure.

Before this plugin, Hermes users had no way to:

• Track spending per cron job or messaging platform
• Set budget limits that actually pause runaway processes
• Compare cost efficiency between different models
• Get real-time cost alerts before hitting billing limits

Now they can manage their AI spend like a modern SaaS — with dashboards, alerts, and automatic circuit breakers.

✨ Key Features

Real Usage Data (Not Estimates)

Captures actual token counts and costs returned by providers like OpenRouter, OpenAI, and Anthropic. No guesswork.

Multi-Level Budget Enforcement

• Soft warnings at 80% of budget
• Hard tool blocks at 100% (prevents new API calls)
• Cron job pauses for automated workflows
• Scope-specific limits (global, per-cron-job, per-platform)

Rich Analytics via Slash Commands

• /stats — session performance, tool usage, cost breakdowns
• /stats cron week — cron job cost comparison across time
• /stats providers — which providers return real vs estimated data
• /budget — current spending vs limits with visual indicators

Zero Model Awareness

Pure observability layer — captures everything through hooks without affecting model behavior or adding latency.

📊 Screenshots

Session Analytics (/stats)

Budget Status (/budget)

Cron Job Cost Comparison (/stats cron week)

Provider Analysis (/stats providers)

🧪 Proof of Concept: Real Data

I tested the plugin with three different models to validate pricing accuracy and budget enforcement:

Budget enforcement works. When I set a $0.001 daily limit and ran a cron job, it correctly paused at $0.18 spending. When I raised the limit to $2.00, jobs resumed normally.

Real provider data. OpenRouter returned actual token counts (Est% = 0%), not estimates. The plugin correctly captured and priced these.

🏗️ Technical Implementation

Hook Pipeline Architecture

on_session_start → pre_api_request → ★ post_api_request → post_tool_call
                                     │
                                     ▼
                               [capture usage]
                                     │
                                     ▼
pre_llm_call (budget check) → pre_tool_call (tool gate) → SQLite storage

Data Layer

• SQLite WAL database — efficient, local, no external deps
• Custom pricing.yaml — override provider rates for accurate cost calculation
• budget.yaml configuration — flexible limits (daily/monthly, global/scoped)
• 94 comprehensive tests — full coverage of edge cases and enforcement logic

Provider Compatibility

Works with any provider that follows the Hermes Agent provider interface:

• ✅ OpenRouter (tested with real usage data)
• ✅ OpenAI (pricing table included)
• ✅ Anthropic (pricing table included)
• ✅ Custom providers (via pricing.yaml overrides)

🎯 Production Ready

This isn't a demo — it's production infrastructure. The plugin includes:

• Error handling — graceful fallbacks when providers return no usage data
• Hot-reload — update budgets via /budget set without restart
• Concurrent safety — SQLite WAL mode handles multiple sessions
• Memory efficiency — hook pipeline adds negligible overhead
• Comprehensive logging — debug telemetry issues with structured logs

🚀 Installation & Usage

1. Install

cd ~/.hermes/plugins
git clone https://github.com/nujovich/hermes-telemetry.git
# Add 'hermes-telemetry' to plugins.enabled in config.yaml
# Restart gateway: hermes gateway restart

2. Configure Budget (Optional)

# Set daily budget
hermes> /budget set global daily 5.00

# Check status  
hermes> /budget

3. Monitor Usage

# Session stats
hermes> /stats

# Cron job breakdown
hermes> /stats cron week

# Provider analysis
hermes> /stats providers

That's it. The plugin immediately starts capturing usage data for all sessions and cron jobs.

🏆 Why this is a win-win

This plugin solves a universal need in AI systems — cost visibility and control. Every Hermes Agent deployment, from personal automation to enterprise cron jobs, benefits from this infrastructure.

It's not just useful, it's essential. Without budget controls, a misconfigured cron job with an expensive model can cost hundreds of dollars overnight. This plugin prevents that.

Real-world tested. I built, deployed, and validated this with actual usage data across multiple providers and models. It's not a concept — it's working infrastructure that saves money and provides operational insight.

Community impact. This sets a standard for observability in the Hermes ecosystem. Other plugin authors can build on these patterns, and users get immediate operational confidence.

📋 Technical Details

• Repository: https://github.com/nujovich/hermes-telemetry
• Documentation: Complete README with architecture, configuration, and troubleshooting
• Tests: 94 passing tests covering all major functionality
• License: MIT
• Dependencies: PyYAML only (for config files)

👨‍💻 About the Author

I'm Nadia Ujovich.

I understand the operational challenges of running AI systems at scale, and I built this plugin to solve the observability gap I see in every deployment.

This plugin makes Hermes Agent production-ready for cost-conscious deployments. It's the infrastructure piece that every serious AI system needs, but few teams build themselves.

Give your agents the observability they deserve. Try hermes-telemetry today.

Made with ☕ for the Hermes Agent ecosystem