The 15 bugs AI coding assistants generate over and over (...

AI coding assistants are fast. They're also surprisingly consistent at making the same class of structural mistakes.

After scanning hundreds of AI-generated files, I kept seeing the same patterns:

# Pattern 1: MISSING_WRITE
# AI generates a save function that never actually saves
def save_user(data):
    validate(data)
    return {"status": "saved"}   # no INSERT, no UPDATE, nothing

# Pattern 2: FAKE_ASYNC
# async keyword with no await anywhere
async def fetch_data(url):
    return requests.get(url)     # synchronous, blocks the event loop

# Pattern 3: STUB_SKELETON
# Placeholder that looks complete but does nothing
def analyze_sentiment(text):
    return {}                    # zero logic

These aren't random bugs. They're structural patterns that appear across languages and models — GPT-4, Claude, Gemini, Copilot. The AI writes code that looks correct at a glance but breaks at runtime.

The problem: existing scanners weren't designed for this. Bandit and Semgrep catch security vulnerabilities. They don't check whether your save_user() actually saves.

What I built

AINAScan — a deterministic AST scanner with:

15 vibe-coding patterns (the structural bugs above)
33 security patterns (SQL injection, SSRF, path traversal, command injection, XSS, etc.)
9 languages: Python, JS, TS, Go, Ruby, Java, PHP, Kotlin, C/C++
No LLM involved — same code always produces the same result

The 15 vibe-coding patterns

Pattern	What it catches
`MISSING_WRITE`	save/store function with no DB write
`FAKE_ASYNC`	async def with no await
`STUB_SKELETON`	function that just returns `{}` or `None`
`DEAD_CALL_RESULT`	calls 3 services, ignores all return values
`HARDCODED_TABLE`	40-key dict replacing what should be a DB query
`INPUT_OUTPUT_DISCONNECTED`	params never used in function body
`TRIVIAL_IF_CHAIN`	7+ elif branches with no DB lookup
`MOCK_PATTERN`	MagicMock in production code
`EMPTY_EXCEPT`	except: pass swallowing errors silently
`MISSING_ERROR_HANDLING`	external API calls with no try/catch
`TRIVIAL_ASSERT`	assert True in tests
`TODO_PLACEHOLDER`	TODO/FIXME left in production
`PARAM_SHADOW`	parameter shadowed by local variable
`SHORT_PASSTHROUGH`	wrapper that adds no value
`CONST_SQL_NO_PARAM`	SQL WHERE with hardcoded value

Try it (30 seconds, no signup)

curl -X POST https://pleasing-transformation-production-90c2.up.railway.app/v1/scan \
  -H "X-API-Key: vg_free_test" \
  -F "file=@your_agent.py"

Response looks like:

{
  "passed": false,
  "block_count": 2,
  "warn_count": 1,
  "issues": [
    {
      "kind": "MISSING_WRITE",
      "severity": "BLOCK",
      "line": 12,
      "detail": "function 'save_user' claims to save but contains no DB write call"
    }
  ]
}

GitHub Action — catch vibe-coding bugs in PRs

name: VibeGuard Security Scan
on: [pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Moonsehwan/aina-vibeguard-action@v1
        with:
          api-key: ${{ secrets.VIBEGUARD_KEY }}
          fail-on-block: 'true'

False positive design

One thing I spent a lot of time on: upstream sanitizer detection.

Before flagging a path traversal issue, the scanner checks 60 lines before the sink for guard patterns:

if '..' in path:
    return 400              # scanner sees this
filepath = open(path)       # and downgrades BLOCK to WARN

This cut the false positive rate on well-maintained open source repos (Django, FastAPI, celery) from ~40% down to near zero.

I also tested on 10 repos with 100k+ GitHub stars — 0 false positives on legitimate code.

推荐订阅源

DEV Community