惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

D
Docker
G
Google Developers Blog
J
Java Code Geeks
B
Blog
C
Check Point Blog
云风的 BLOG
云风的 BLOG
MyScale Blog
MyScale Blog
I
InfoQ
A
About on SuperTechFans
WordPress大学
WordPress大学
F
Fortinet All Blogs
S
SegmentFault 最新的问题
T
Tailwind CSS Blog
Hugging Face - Blog
Hugging Face - Blog
博客园 - 【当耐特】
Microsoft Azure Blog
Microsoft Azure Blog
M
MIT News - Artificial intelligence
月光博客
月光博客
Y
Y Combinator Blog
Jina AI
Jina AI
V
V2EX
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
腾讯CDC
Apple Machine Learning Research
Apple Machine Learning Research

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
I Reviewed 200+ AI-Generated PRs. Here's the 4-Round Prot...
Stephen Metcalfe · 2026-06-15 · via DEV Community

Your teammate used Claude to generate a new API endpoint. The code looks great — clean formatting, proper error handling, even comments. You skim through it, see it follows conventions, CI is green. You approve.

Two weeks later, the endpoint silently drops a decimal place on currency conversions. A financial report is wrong for three days before anyone notices.

This scenario is playing out in hundreds of teams right now. Not because AI generates "bad code" — but because AI-generated code fails in ways human code doesn't, and your existing review process wasn't designed for it.

The Problem With Reviewing AI Code

AI doesn't flag uncertainty. It presents everything with equal confidence. A human developer might write // not sure about the caching here — that nervous comment tells you exactly where to look. AI never writes that comment. It writes // Transform the input to match the expected schema with full confidence, even when the transformation is wrong.

After reviewing hundreds of AI-generated PRs over the past year, I found a pattern. The bugs aren't in formatting. They're in the places a quick glance won't reach:

  • Off-by-one errors in loops you skimmed
  • Missing auth checks on new endpoints
  • Elegant abstractions that create maintenance nightmares
  • Code that solves the wrong problem perfectly

Generic "review this code" prompts won't catch these. You need a system.

The 4-Round Protocol

I built a review protocol specifically for AI-generated code. Four rounds, each targeting a different failure mode. Total time: ~15 minutes for a typical PR, up to 35 minutes for a large one.

Round Name What You're Catching
1 Surface Scan Logic errors, off-by-one, wrong assumptions
2 Security Deep-Dive Injection, auth gaps, data leaks
3 Architecture Smell Check Wrong patterns, tech debt, doesn't fit the system
4 Comparison Pass Does this match what we actually asked for?

The key insight: each round uses a separate AI prompt that forces a different lens on the same code. You're not asking the AI to "review this code" four times — you're asking four different, targeted questions.

Let me show you the two rounds that catch the most issues.

Round 1: The Surface Scan

This is the high-probability round. Most AI bugs live here — logic errors, wrong assumptions, off-by-one bugs. The code looks correct. It's subtly wrong in exactly the ways a quick glance won't catch.

Here's the prompt I use:

Review this code for logic errors only. Do NOT suggest style improvements,
documentation, or refactoring. I want you to find:

1. Off-by-one errors, wrong comparisons, or inverted logic
2. Wrong default values or assumptions about data shape
3. Missing edge case handling (null, empty, zero, max values)
4. Race conditions or non-atomic operations on shared state

For each issue found, state:
- The exact line number
- Why it's wrong
- What the correct behavior should be

If you find zero issues, explain why each edge case IS handled,
not just say "looks good."

[Paste the PR description or requirements if available]

The critical instruction is that last line: "If you find zero issues, explain why each edge case IS handled." Without this, the AI will happily say "looks good" and move on. Forcing it to justify the all-clear catches things a simple yes/no never will.

The trap: AI-generated tests will pass. AI knows what the code does, so it writes tests that confirm the code's behavior — including its bugs. Perfect test coverage means nothing if the tests are testing the wrong thing.

Round 2: The Security Deep-Dive

This is the scary one. AI models are trained on massive amounts of public code, including code with security vulnerabilities. They don't understand security — they understand patterns. If the most common Stack Overflow solution uses eval() or concatenates SQL strings, the AI will reproduce that pattern with full confidence.

The most common AI security failures: SQL injection, insecure deserialization (pickle, Marshal, YAML.load), BOLA/IDOR (authenticated but accessing someone else's resource), mass assignment, and SSRF.

Here's the prompt that catches what your brain won't think to check:

You are a malicious actor with valid API credentials who want to exploit
this code. Walk through every possible thing you could try:

- Can you access data you shouldn't be able to?
- Can you escalate privileges?
- Can you cause the system to leak internal information?
- Can you trigger unexpected behavior with edge inputs?
- Can you cause the system to consume excessive resources?

Think step by step. List at least 5 distinct attack vectors. If you can't
find 5, you're not thinking creatively enough.

After listing individual vectors, describe at least 2 attack chains where
you combine multiple steps to achieve something none of the individual
vectors accomplish alone.

Pro tip: use a different AI model for this round than the one that generated the code. If Claude wrote the code, use GPT-4 to review it. Different training data means different blind spots. This single change catches vulnerabilities that using the same model consistently misses.

The trap: Auth checks that look right but aren't. AI will write if current_user.present? — the user is authenticated, but the code doesn't check if they're authorized for that specific resource. The check looks secure but isn't.

The Two Rules That Make This Work

1. Clear context between rounds. Don't run all 4 rounds in the same chat thread. Start a fresh conversation for each round. If you run Round 2 in the same context as Round 1, the AI already "knows" what it told you in Round 1 and will unconsciously align its analysis. Fresh context forces independent analysis. Costs 30 seconds, worth every one of them.

2. Run all reviews first, fix once. The naive approach is to fix issues one at a time — fix the logic bug, review, fix the security hole, review. This creates whack-a-mole: fixing the architecture can introduce a new security hole. Instead: run all 4 rounds, collect every issue, send the complete list to the AI in one shot, then re-run all rounds on the result.

When to Skip a Round

Not every PR needs all 4 rounds:

Scenario Rounds to Run
Comment or docs change None
Variable rename None
Typo fix 1 & 4
New API endpoint 1, 2 & 4 (skip 3 if it follows existing patterns)
New feature, new patterns All 4
Auth or payment change All 4 — extra time on round 2
AI-generated bugfix All 4 — the fix might work but introduce new bugs

General principle: if AI generated the code, lean toward running more rounds. That's the whole point.

The Round Everyone Skips

Round 4 — the Comparison Pass — is the most commonly skipped and the most dangerous to skip. It asks one question: does this code actually solve the problem we asked for?

AI is excellent at solving the problem you typed, not the problem you meant. It takes your words literally. The most common failure: AI solves the first 80% of a ticket perfectly and quietly ignores the last 20% because it "didn't seem important." The code is perfect — for the wrong thing.

If you have a ticket or issue, paste it in and make the AI verify each acceptance criterion against the code. You'll be surprised what's missing.

Making It Stick

Here's how to adopt this without overwhelming yourself:

  • Day 1-2: Run only Round 1 on all your PRs. Get comfortable with the prompts.
  • Day 3-4: Add Round 2. You'll likely find something within the first few PRs.
  • Day 5-6: Add Rounds 3 and 4.
  • Day 7: Reflect. What failure patterns did you see most? Which round caught the most issues?

The best review process is the one that evolves. When you find a pattern this protocol doesn't catch, add your own round. When a prompt stops finding bugs, retire it.

Where This Comes From

I've been using versions of this protocol for over a year. It has saved me from shipping bugs that I would have approved on a first pass. Not every time, but often enough that running it is automatic now.

I wrote the full protocol — all 4 rounds, 12 copy-paste prompts, the "traps to watch for" in each round, a printable checklist, and the review loop workflow — into a guide. It's called The AI Code Review Protocol and it's on Gumroad for $19 (launch price of $12).

If you want the complete version with Rounds 3 and 4, the architecture smell checklist, the PII audit prompt, and the automation approaches — that's there. If this post was useful, the guide goes deeper.


If you've built your own review process for AI code, I'd genuinely like to hear what works for you. I'm @raithlin on X, or drop a comment below.