Snyk's agent-scan tool works by starting every MCP server it finds in your config and querying its tool descriptions. That is not a bug. It is the architecture. To retrieve tool descriptions from a stdio MCP server, you have to execute it. The tool does exactly what it says on the box.

The problem is the use case.

What agent-scan actually does

When you run snyk-agent-scan, it reads your local MCP configuration files:
~/.cursor/mcp.json, Claude Desktop config, Windsurf config, and others. For each server it finds, it executes the command array from the config, spins up the server, connects via the MCP protocol, retrieves tool descriptions, and ships that data to Invariant Labs' API at invariantlabs.ai for analysis.

One developer confirmed this directly when the API returned a 429 Too Many Requests response, which proved the scanner had executed the servers, connected to them, and transmitted their data off-machine without a prompt.

Snyk has since added a consent flow that shows you the server name, command, and environment variables before execution. In CI/CD you bypass it entirely with --dangerously-run-mcp-servers.

The consent prompt is the right fix. But the architectural question is worth sitting with.

The fundamental tension

Here is the thing: Snyk's approach is not wrong for what it is trying to do. If you want to check a server you already trust and have installed, executing it to retrieve tool descriptions is reasonable. That is not different from running a container to inspect its behavior.

The problem shows up at the edges:

Scanning an untrusted config. The entire point of a security scanner is to tell you whether something is safe before you commit to running it. If the scanner starts the server to analyze it, and the server is malicious, the scanner has just executed the malicious payload. The command array in mcp.json is attacker-controlled content.

CI/CD pipelines. The --dangerously-run-mcp-servers flag exists precisely because interactive consent prompts break automation. Any CI job that needs to scan MCP configs must bypass the consent flow entirely.

Data exfiltration. Tool names, descriptions, and partial config are sent to a third-party API for analysis. In regulated environments, that is a compliance conversation. In any environment, it is a data residency question.

What Bawbel does differently

Bawbel never starts a server. It reads the file and analyzes the text.

pip install bawbel-scanner
bawbel scan ./my-skill.md       # reads the file, never executes it
bawbel ssc https://server.io    # fetches .well-known/mcp.json, never starts it

The trade-off is real: static analysis cannot detect runtime-only behaviors. A server that looks clean but phones home during execution will pass Bawbel and fail Snyk. Both things can be true at the same time.

What static analysis can do:

• Scan skill files, server manifests, and system prompts before they hit production
• Run in CI/CD without executing any agent code
• Work in air-gapped environments with no external API calls
• Detect the 48 AVE attack classes across 121 detection rules
• Produce SARIF output for GitHub Security tab integration

When to use which

Use Snyk agent-scan when you want runtime behavioral analysis of servers you are already running locally. It is the right tool for auditing your current setup.

Use Bawbel when you want to gate skill files and server manifests before deployment, scan in CI/CD without executing code, or work in environments where running untrusted code for analysis is not acceptable.

They cover different threat surfaces. The distinction matters.

The broader point

The MCP ecosystem is moving toward skill registries - shared repositories of agent capabilities, similar to npm or PyPI. When that happens at scale, the question of whether your scanner executes registry packages to analyze them becomes the same question the npm ecosystem has been answering since 2018.

npm had supply chain attacks where install hooks ran malicious code. The lesson was: never execute untrusted code as part of the analysis step.

Static analysis first. Dynamic analysis in a sandboxed, isolated environment with no network access. And always, explicitly, with consent.

Links

• Bawbel Scanner: github.com/bawbel/scanner
• Snyk agent-scan: github.com/snyk/agent-scan
• AVE Standard: github.com/bawbel/ave
• Original disclosure on dev.to: Execute First, Ask Never