惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Hacker News: Ask HN
Hacker News: Ask HN
WordPress大学
WordPress大学
H
Help Net Security
小众软件
小众软件
N
Netflix TechBlog - Medium
C
Check Point Blog
量子位
Last Week in AI
Last Week in AI
GbyAI
GbyAI
Martin Fowler
Martin Fowler
M
MIT News - Artificial intelligence
博客园 - 聂微东
Engineering at Meta
Engineering at Meta
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
J
Java Code Geeks
D
DataBreaches.Net
Project Zero
Project Zero
P
Proofpoint News Feed
T
Threat Research - Cisco Blogs
Security Latest
Security Latest
Cisco Talos Blog
Cisco Talos Blog
Recorded Future
Recorded Future
I
Intezer
L
Lohrmann on Cybersecurity
Cyberwarzone
Cyberwarzone
博客园_首页
C
Cyber Attacks, Cyber Crime and Cyber Security
L
LangChain Blog
P
Palo Alto Networks Blog
V
V2EX
D
Darknet – Hacking Tools, Hacker News & Cyber Security
T
The Exploit Database - CXSecurity.com
The Hacker News
The Hacker News
Blog — PlanetScale
Blog — PlanetScale
G
GRAHAM CLULEY
T
The Blog of Author Tim Ferriss
C
Cisco Blogs
The Register - Security
The Register - Security
L
LINUX DO - 热门话题
P
Privacy & Cybersecurity Law Blog
Scott Helme
Scott Helme
F
Full Disclosure
博客园 - 司徒正美
Recent Announcements
Recent Announcements
IT之家
IT之家
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
Attack and Defense Labs
Attack and Defense Labs
Cloudbric
Cloudbric
Help Net Security
Help Net Security
The Last Watchdog
The Last Watchdog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Why Your AI-Generated Code Passes Tests but Fails Production
Ashwini Kuma · 2026-05-19 · via DEV Community

AI coding assistants are fast — but fast in the wrong direction is just a faster way to fail. Here's how specs and BDD scenarios turn AI into a reliable executor, not a fast guesser.

The views and opinions expressed in this article are solely our own and do not represent those of our employer, Amazon, or any of its affiliates. All content is based on publicly available information and our personal experience. Any references to products or tools are based solely on publicly available information. No confidential or proprietary information has been disclosed.


AI coding assistants have changed the economics of software development. A well-prompted AI can produce in an hour what used to take a day. Teams are shipping faster than ever. And a growing number of those teams are shipping the wrong thing faster than ever.

The problem isn't the AI. The problem is what you give it to work with.


AI Is Only as Good as Its Context

When you ask an AI to implement a feature with a vague prompt, it does exactly what you'd expect — it makes reasonable assumptions, fills in the gaps with general software patterns, and produces code that looks correct, compiles cleanly, and is wrong for your domain in ways that won't surface until production.

This isn't a flaw in the AI. It's a structural reality. AI generates code based on pattern recognition. It doesn't know your business rules. It doesn't know that a missing rent estimate should return a score with confidence: "low" instead of throwing an error. It doesn't know that negative cash flow shouldn't automatically fail a deal — it should reduce the score by a maximum of 30 points. It fills those gaps with the most statistically likely answer, which is plausible, passes a surface-level review, and violates your domain logic.

The solution isn't better prompting. It's giving the AI a contract to implement against.

That contract is a spec. The mechanism that enforces it is a BDD scenario suite. Together, they transform AI from a fast guesser into a fast executor — and that distinction is everything.


The Contract: Spec-Driven Development

A spec is not a PRD. It's not a Jira ticket. It's a precise, unambiguous description of what a system must do — inputs, outputs, rules, edge cases, error states — written before any code exists.

When an AI reads a spec, it stops guessing. Every rule is explicit. Every edge case is named. The AI has no gaps to fill because the spec filled them first.

Feature: Deal Score Calculation
  Input: property price, rent estimate, assumptions (rate, down payment, taxes)
  Output: score 0–100, grade A–F, confidence level

  Rules:
  - Grade maps: 80+  A, 65–79  B, 50–64  C, 35–49  D, <35  F
  - Missing rent estimate → score still computed, confidence: "low"
  - Negative cash flow reduces score by max 30 points, not automatic failure
  - Price above $800k  flag "high price tier", no score penalty

Enter fullscreen mode Exit fullscreen mode

Compare what an AI produces with and without this spec:

Prompt without spec: "implement deal score calculation"

AI output:
- Returns 0 for missing rent estimate          ← wrong
- Fails immediately on negative cash flow      ← wrong
- No price tier flagging                       ← missing
- Passes all generic unit tests                ← tests share the same blind spots

Enter fullscreen mode Exit fullscreen mode

Prompt with spec: "implement deal score calculation per this spec: [spec above]"

AI output:
- Handles missing rent estimate, returns confidence: "low"   ✓
- Caps cash flow penalty at 30 points                        ✓
- Flags high price tier without penalizing score             ✓
- Grade mapping matches spec exactly                         ✓

Enter fullscreen mode Exit fullscreen mode

Same AI. Same tool. The spec is the only variable. The spec didn't slow the AI down — it directed it. The AI's speed is now working for you instead of against you.


The Guardrail: BDD Scenarios

The spec tells the AI what to build. BDD scenarios tell the AI — and your CI pipeline — whether it built it correctly.

BDD scenarios are written in plain language (typically Gherkin) that describes system behavior from the outside. They are independent of implementation. The AI cannot game them by writing tests that match its own code, because they were written from the spec before the code existed.

Scenario: Missing rent estimate
  Given a property priced at $450,000
  And no rent estimate is available
  When the deal score is calculated
  Then a score is still returned
  And the confidence level is "low"

Scenario: Negative cash flow
  Given a property with negative monthly cash flow of -$400
  When the deal score is calculated
  Then the score is reduced by no more than 30 points
  And the deal is not automatically failed

Scenario: High price tier
  Given a property priced at $850,000
  And a monthly rent estimate of $4,500
  When the deal score is calculated
  Then the "high price tier" flag is present
  And the score is not penalized for price alone

Enter fullscreen mode Exit fullscreen mode

When you give an AI these scenarios alongside the spec, it has a verifiable definition of done. It can run the scenarios against its own output, catch failures, and self-correct before a human ever sees the code. The scenarios are not just tests — they are the AI's feedback loop.

This is why BDD scenarios are more valuable in an AI-driven workflow than traditional unit tests. Unit tests written by AI share the same assumptions as the code AI writes. BDD scenarios written from the spec are assumption-free. They describe behavior, not internals.


AI as a Spec Stress-Tester

The relationship between AI and specs isn't one-directional. AI is also one of the most effective tools for finding gaps in a spec before implementation begins.

Give an AI your spec and ask it to break it:

Prompt: "Given this spec, what edge cases are missing?

Rules:
- Score is 0–100
- Missing rent estimate → confidence: 'low'
- Negative cash flow reduces score by max 30 points"

AI response:
- What if both rent estimate AND cash flow data are missing simultaneously?
- What if price is $0 or negative — validation error or score of 0?
- What if cash flow is exactly $0 — is that negative or neutral?
- What if compounding penalties push the score below 0?
- Is there a maximum rent estimate that should be flagged as suspicious input?

Enter fullscreen mode Exit fullscreen mode

Every one of those questions is a potential production bug. Answered in the spec, they cost nothing. Discovered after deployment, they cost a sprint, a hotfix, and a post-mortem.> AI-assisted spec review is the highest-leverage use of AI in the entire development cycle — because it eliminates bugs before a single line of code is written.


The Full AI-Driven Workflow

Here's what a feature cycle looks like when AI, specs, and BDD scenarios work together:

Day 1 — Spec authored by humans, stress-tested by AI
  → Tech lead, PM, and senior engineer write the spec
  → AI is asked: "what edge cases are missing from this spec?"
  → Gaps are resolved before implementation begins
  → Spec committed to the repo alongside the code it governs

Day 2 — Scenarios written from the spec
  → Engineers and QA convert spec rules into BDD scenarios
  → AI generates adversarial scenarios: "write scenarios that would
     catch a naive implementation"
  → Every rule has at least one scenario; every edge case has one too

Day 3 — AI implements against spec + scenarios
  → AI is given spec and scenarios as context
  → AI implements to pass scenarios, not to satisfy a vague description
  → AI runs scenarios locally, self-corrects, opens PR with green scenarios

Day 4 — Human review, focused on architecture
  → Reviewer sees passing scenarios — behavioral correctness is already verified
  → Review focuses on approach, performance, and maintainability
  → Review is faster because the question "does this do the right thing?" 
     is already answered

Day 5 — CI enforces the contract forever
  → Scenarios run on every commit
  → Any future AI-generated change that breaks behavior fails the build
  → The spec is permanently enforced without anyone having to remember it

Enter fullscreen mode Exit fullscreen mode

The human's role in this workflow shifts toward high-judgment work: writing specs, reviewing architecture, resolving edge cases. The high-volume work — implementation, test generation, gap analysis — goes to AI. That's the right division of labor.


Why AI-Generated Tests Are Not Enough

A common mistake teams make when adopting AI tools is asking the AI to write tests alongside the implementation. This feels productive. It produces coverage numbers. It doesn't protect you.

AI-generated tests share the same assumptions as AI-generated code. If the AI misunderstood the spec (or there was no spec), the tests will validate the wrong behavior confidently. You end up with a green test suite that proves the code does what the AI thought you wanted, not what you actually needed.

BDD scenarios written from the spec before implementation are the only tests that are truly independent of the AI's assumptions. They describe what the system must do from the outside. The AI cannot write them correctly without the spec — and if it tries, the gaps in the spec become immediately visible.

// AI-generated unit test (no spec):
it('returns 0 when rent estimate is missing')  // ← validates wrong behavior

Enter fullscreen mode Exit fullscreen mode

# BDD scenario (from spec):
Given no rent estimate is available
When the deal score is calculated
Then a score is still returned        # ← catches the wrong behavior
And the confidence level is "low"

Enter fullscreen mode Exit fullscreen mode

The BDD scenario catches exactly what the AI-generated test missed. That's not a coincidence — it's the structural difference between tests that describe implementation and tests that describe behavior.


The Compounding Advantage

The real payoff of this approach isn't in a single feature cycle. It's what happens to the codebase over time.

Every feature built with a spec and BDD scenarios adds a permanent behavioral contract to the codebase. Six months later, when an AI is asked to refactor the scoring engine, the scenarios tell it immediately whether the behavior survived. The AI doesn't need to read the original spec, find the original engineer, or guess at intent. It runs the scenarios. They pass or they don't.

This is the compounding advantage: every spec and scenario written today makes every future AI interaction with that code safer and faster. The codebase becomes a set of contracts that AI can execute against with confidence, rather than a codebase it has to interpret and guess at.

Teams that build this way are not just shipping faster today. They are making every future sprint faster too.


The Practical Setup

The tooling to run this workflow end to end exists today:

Spec files: Markdown committed to the repo, co-located with the code they govern. AI reads them as context on every implementation prompt.

Scenario files: Gherkin syntax, run by Cucumber (JS/Java/Ruby), pytest-bdd (Python), or SpecFlow (.NET). Committed to the repo, run in CI on every commit.

AI tools: Claude Code, GitHub Copilot, Cursor — any assistant that accepts file context. The spec and scenario files are the context. Quality of output scales directly with quality of spec.

CI enforcement: GitHub Actions, AWS CodePipeline, or any CI system. Failing scenario blocks the merge. No exceptions, no overrides.

The spec and scenarios are not documentation overhead. They are the interface between human intent and AI execution. Remove them and AI is fast and unpredictable. Keep them and AI is fast and contract-bound.


The Rule That Makes It Work

One rule, enforced without exception:

🚫 No scenario, no code.

If a behavior isn't described in a scenario, it doesn't get implemented — by a human or an AI. If the AI wants to handle a case the spec didn't cover, the scenario gets written first. That either updates the spec or surfaces a disagreement that gets resolved before the code exists.

This rule feels like friction. It is friction — once, upfront, where it costs almost nothing. It eliminates the friction that compounds: the rework, the regression, the hotfix, the post-mortem, the sprint lost to "that's not what I meant."


Closing

AI coding tools are not going to slow down. The teams that figure out how to direct them precisely will compound their advantage over the teams that are still prompting vaguely and reviewing hopefully.

The spec is the precision. The BDD scenarios are the enforcement. Together they give AI exactly what it needs to stop guessing and start executing — against a contract that humans defined, scenarios verify, and CI protects on every commit forever.

Fast AI with no contract is a liability. Fast AI with a contract is a force multiplier.

Write the spec first. Write the scenarios second. Let the AI do the rest.


About the Authors

**Written by Ashwini Kumar and Malcolm Jones — Software Engineers @ Amazon.

The views and opinions expressed in this article are solely our own and do not represent those of our employer, Amazon, or any of its affiliates.