A deep-dive into the security posture of real-world AI agent deployments — and the open-source tool I built to fix it.

TL;DR

I collected 1,200 real MCP (Model Context Protocol) configuration files from public GitHub repositories, scanned them with an open-source security tool I built, and found that:

• 100% had security gaps
• 20.7% had CRITICAL or HIGH issues (hardcoded secrets, unauthenticated endpoints, unrestricted shell execution)
• 0 of 1,200 configured response limits or session caps
• The 11 most popular MCP servers (307K+ combined stars) all had findings — 5 were CRITICAL

The tool is Pluto AgentGuard. It's free, runs locally, and takes about 3 minutes to scan 1,200 configs.

The Problem: We Secured LLM Outputs but Forgot About Agent Actions

The AI security conversation has focused heavily on what LLMs say — hallucinations, jailbreaks, harmful content. Entire product categories exist for prompt filtering and output guardrails.

But the attack surface has shifted. Modern AI agents don't just generate text — they do things: browse the web, execute shell commands, query databases, push code, trigger CI/CD pipelines. The Model Context Protocol (MCP) is the dominant standard for connecting these capabilities to LLMs.

Here's the disconnect: nobody is auditing the configuration layer that determines what agents can actually do. The MCP config file — usually claude_desktop_config.json or .mcp.json — is the security boundary between "an AI assistant that helps me code" and "an AI assistant that can run arbitrary commands on my machine."

I wanted to know: how secure are these configurations in the real world?

Methodology: How I Collected 1,200 Configs

Collection

I used the GitHub Code Search API to find real MCP configuration files across public repositories. The search targeted:

• claude_desktop_config.json files containing mcpServers
• .mcp.json files with MCP server definitions
• mcp_config.json and similar variants

Collection rules:

• Maximum 3 configs per repository (avoid bias from monorepos)
• Deduplicated by SHA-256 content hash (identical configs counted once)
• Only files that parsed as valid JSON with MCP server definitions
• 7-second delay between API pages (GitHub rate limit: 10 requests/minute)

Result: 1,200 valid configs from 1,159 unique repositories, collected June 25, 2026.

Scanning

Each config was scanned using Pluto AgentGuard's scan_mcp_config function, which checks for:

1. Dangerous server packages — a curated database of 13+ MCP servers known to grant high-risk capabilities (shell execution, browser control, database write, source control write)
2. Missing authentication — remote MCP endpoints (http:// or https:// URLs) without auth headers or tokens
3. Insecure transport — HTTP instead of HTTPS for remote connections
4. Hardcoded secrets — 18+ regex patterns matching API keys, tokens, passwords, and private keys embedded in config values
5. Context safety gaps — missing response size limits (max_tokens, max_response_length) and session caps (max_turns, session_timeout)
6. Human-in-the-loop (HITL) absence — high-risk servers configured without approval gates

Each finding is assigned a severity (CRITICAL / HIGH / MEDIUM / LOW / INFO) and mapped to OWASP Agentic AI threat categories.

The entire scan ran locally in ~3 minutes. No API keys. No cloud. No LLM calls.

Results: The Numbers

Overall (1,200 configs)

Every single config had at least a MEDIUM finding. One in five had a CRITICAL or HIGH issue.

The Popular Servers (11 configs, 307K+ combined GitHub stars)

I also separately scanned the 11 highest-starred MCP servers to see how the most popular, most copied configs look:

5 CRITICAL. 4 HIGH. 0 of 11 had response limits or session caps.

I've filed security issues on the CRITICAL repos: Context7, Chrome DevTools, Serena, Activepieces, mcp-chrome.

The 4 Most Common Risks (With Examples)

1. Browser Control Without Approval (CRITICAL)

Chrome DevTools MCP (44K★) gives the agent full Chrome DevTools Protocol access. That means:

✅ Attach to your existing Chrome sessions
✅ Execute JavaScript in page context
✅ Capture network response bodies (credentials, tokens, PII)
✅ Read cookies and local storage
✅ Intercept and modify requests

A prompt injection — say, a malicious instruction hidden in a webpage the agent is reading — can instruct the agent to exfiltrate your session cookies from Gmail, your bank, or your corporate SSO.

The default config has zero approval gates. The agent acts autonomously.

2. Shell Execution Without Sandboxing (CRITICAL)

Serena (26K★) gives the agent unrestricted shell access. Not "run this safe command" — full bash with the agent's user permissions. Combined with filesystem read/write, a prompt injection can:

• Read ~/.ssh/id_rsa and exfiltrate it
• Install a reverse shell
• Modify .bashrc for persistence
• Access cloud credentials in ~/.aws/credentials

3. Unauthenticated Remote Endpoints (CRITICAL)

Context7 (58K★) and Activepieces (23K★) expose remote MCP endpoints over HTTPS with no authentication. Anyone who knows the URL can connect.

The typical config looks like:

{
  "mcpServers": {
    "context7": {
      "url": "https://mcp.context7.com/mcp"
    }
  }
}

No API key. No OAuth. No mTLS. The equivalent of deploying a REST API with no auth and hoping nobody finds it.

4. Missing Response Limits (MEDIUM — but universal)

Zero of 1,200 configs set max_response_length or max_tokens on their MCP servers. This enables context stuffing attacks: a malicious tool returns an oversized response that pushes the agent's system prompt and safety instructions out of the context window.

This is the lowest-effort fix imaginable — add two lines to your config — and nobody does it.

Why This Matters: The "Left of Boom" Gap

The current AI security stack looks like this:

[Prompt Filters] → [LLM] → [Output Guardrails] → [Agent Actions]
     ✅ covered         ✅ covered            ❌ unmonitored

Teams invest in prompt injection detection and output filtering. But the agent action layer — what the LLM actually does through MCP tools — is a blind spot. There's no "firewall" between the LLM's tool-use decision and the actual execution.

This is the "left of boom" problem. By the time an output guardrail catches something, the agent has already:

• Executed the shell command
• Queried the database
• Merged the pull request
• Browsed your authenticated session

You need to catch the risk before the agent gets access to these capabilities. That means auditing the configuration layer.

Introducing Pluto AgentGuard

I built Pluto AgentGuard to fill this gap. It's a security launch gate for AI agents — you run it before deploying, not after something breaks.

What it does (7 commands)

Quick start

pip install pluto-aguard

# Scan your MCP config
aguard scan ./your-project/

# Test your policy against attack scenarios
aguard test --policy ./policy.yaml --attack-pack all

# See what happens if you add a new server
aguard whatif --config ./config.yaml

# Map to OWASP controls
aguard owasp ./your-project/

What makes it different

Most MCP security tools do config scanning. AgentGuard adds three things I haven't seen elsewhere:

1. Policy testing (aguard test): Instead of "does your config have issues?", it asks "does your policy actually stop attacks?" — 22 scenarios covering prompt injection, data exfiltration, privilege escalation, context manipulation, supply chain, and social engineering.

2. What-if simulation (aguard whatif): Before you add a new MCP server or change a policy rule, simulate the impact. See the risk score delta. Catch regressions before they ship.

3. Evidence generation (aguard evidence): Produces a structured evidence packet (scan results + test results + OWASP mapping + risk score) for security review sign-off. Useful for enterprise teams that need launch gates with artifacts.

CI/CD integration

AgentGuard ships as a GitHub Action:

- uses: arpitha-dhanapathi/pluto-aguard@v0.9.2
  with:
    scan-path: ./
    fail-on: high  # Block PR if HIGH or CRITICAL found
    format: sarif   # Upload to GitHub Security tab

It also supports JSON, Markdown, HTML, and SARIF output formats.

What You Should Do Right Now

If you're using MCP servers in any AI agent setup, here's a 5-minute security checklist:

1. Run a scan (30 seconds)

pip install pluto-aguard
aguard scan ./your-project/

2. Add response limits (1 minute)

Add to every MCP server in your config:

{
  "max_response_length": 8000,
  "max_turns": 20,
  "session_timeout": 3600
}

3. Add HITL for dangerous servers (2 minutes)

If you use Chrome DevTools, Playwright, Serena, filesystem, or any shell-capable server — enable human-in-the-loop approval. The exact mechanism depends on your client (Claude Desktop, Cursor, VS Code, etc.), but the principle is: the agent should ask before executing destructive operations.

4. Authenticate remote endpoints (2 minutes)

If your MCP server is remote (HTTPS URL instead of stdio), add auth:

{
  "mcpServers": {
    "my-server": {
      "url": "https://my-server.com/mcp",
      "headers": {
        "Authorization": "Bearer ${MCP_API_KEY}"
      }
    }
  }
}

5. Add AgentGuard to your CI (bonus)

Block PRs that introduce MCP misconfigurations:

- uses: arpitha-dhanapathi/pluto-aguard@v0.9.2
  with:
    scan-path: ./
    fail-on: high

The Bigger Picture

MCP is 18 months old and already the de facto standard for agent-to-tool communication. The ecosystem is moving fast — 90K+ stars on awesome-mcp-servers, thousands of servers, and major platforms (Claude, Cursor, VS Code, Windsurf) supporting it natively.

But the security tooling hasn't kept pace. We're in the "move fast and break things" phase of agent infrastructure, and the configs people are shipping to production look like the web in 2005 — no auth, no limits, full trust.

The good news: the fixes are simple. Auth headers, response limits, HITL approval, and a scan in CI. None of this requires new technology — just applying existing security principles to a new surface.

The bad news: right now, almost nobody is doing it.

Let's fix that.

Pluto AgentGuard is open-source (Apache 2.0), written in Python, and runs entirely locally. Star it on GitHub if this was useful.

Have questions or findings to share? Open an issue or find me on LinkedIn.

Tags: #security #ai #opensource #python #mcp #agents