惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Hugging Face - Blog
Hugging Face - Blog
量子位
H
Help Net Security
Microsoft Azure Blog
Microsoft Azure Blog
MongoDB | Blog
MongoDB | Blog
小众软件
小众软件
爱范儿
爱范儿
博客园 - 【当耐特】
Vercel News
Vercel News
S
SegmentFault 最新的问题
M
MIT News - Artificial intelligence
F
Fortinet All Blogs
Apple Machine Learning Research
Apple Machine Learning Research
GbyAI
GbyAI
博客园 - 叶小钗
博客园_首页
V
Visual Studio Blog
宝玉的分享
宝玉的分享
B
Blog
MyScale Blog
MyScale Blog
C
Check Point Blog
博客园 - 三生石上(FineUI控件)
L
LangChain Blog
V
V2EX

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Every Test Passed. The User Still Couldn't Play the Game.
Teru Murata · 2026-06-22 · via DEV Community

"Look! Every test is green! The API returns 200 OK!"
"Relax. The system works perfectly. If the user is just standing there staring at the screen, that's a user problem."

I was two years into my first engineering job, and I had quietly decided my senpais were hopeless. They lived inside dashboards, barely touched the actual product, and got cheerfully drunk on coverage numbers. Their one redeeming quality was that the drunker they got on "the code works," the more pleasant they became.

But code working and a real human getting what they came for are two completely different things. A button can return 200 OK and still leave a person staring at an unchanged screen until they give up and leave.

So one afternoon, instead of arguing, I opened a terminal and built a ~30-line shell spell that finds every UX dead-end without running the app even once. I call it the two-agent static walkthrough.

The spell

Two LLM agents, talking to each other in a loop:

  • Agent A — the user. A concrete persona with a concrete goal: "I'm not a programmer. I just want a playable tic-tac-toe I can open and click." Its defining trait is that it is stubborn. It does not quit at the first disappointment — it keeps trying different things.
  • Agent B — the app. But B is forbidden to imagine anything. B is given read access to the real source code and told to trace the actual code path the user's action hits, then narrate what the user would truly experience — citing file:line, and confessing every gap, dead-end, stale state, and lie. No inventing behavior the code does not have.

Nobody launches the app. The two AIs simply walk, turn by turn, down the path of despair a real user would take — and print it to the terminal.

The tragedy: an AI mini-game generator

I aimed it at the feature nobody on the team wanted to touch: an AI mini-game generator. Agent A's goal was simple: press the button and get a playable tic-tac-toe. The merciless log began.

Turn 1 — the button's betrayal
A (user): "Okay! I type 'make me tic-tac-toe' and hit Generate! This is exciting!"
B (app): "Bad news. Per the routing at app.js:45, your request was swallowed by the prototype-era v1/chat endpoint, not the new game-generation API. The new pipeline is never reached."

The tests were written against the new API. The button still pointed at the old one. With no E2E test, nobody noticed.

Turn 2 — the un-clickable void
A: "Huh, a blob of code-looking text appeared. Fine — I'll click it to open the playable preview! click"
B: "Useless. The markdown the old API returned was poured into a plain div, not a preview iframe. There is no link and no handler (render.js:88). Click all you like; nothing will change."

Turn 3 — structural death and a false blessing
A: "Ugh. I don't understand code, but it's broken — so I'll press Fix Game and let the AI repair it!"
B: "Your struggle ends at api.py:112. Repair requires a session_id that only the new API issues; you went through the old one, so it is null. The backend returned a 500: nothing to fix."
A: "What?! So an error shows on screen, right?"
B: "No. The notification logic sits outside the try/catch (app.js:88). So while the backend is dying, your screen proudly displays a green toast: 'Game updated!'"

An error underneath, a success message on top. The single most maddening UX pattern there is.

Turn 4 — the truncated hope
A: "AAAH. It says 'updated' and nothing changed! Fine — I'll copy the code text myself, paste it into an HTML file, and play it by force!"
B: "My condolences. The old API still has a 500-character output cap. The code you are copying is severed just before </html>. It will never run. ...Game over."
A: "......" (leaves)

She kept going — three more doors I'll spare you the full transcript of. She scoped the request down to a single module; she went hunting for a separate goal entrance that the dead "course-correct" button implied must exist somewhere; and finally she asked the app to stop delivering anything and just become the game — draw the board in chat, take her moves. Every one emptied into the same pipeline, behind the same cheerful "working…".

Seven tactics. Every one of them died a structural death behind a 200 OK or a fake success toast — exactly the spots a normal unit test paints green. This is the state of "the code works and the user despairs."

Why it works

  • A stubborn persona exhausts the real paths. My first run let the user quit after one letdown and found almost nothing. The run where A was told "give it a fair, thorough try; only quit when truly dead-ended" found everything. The real despair lives past the first dead-end.
  • B is grounded in real code, so it cannot hallucinate a happy path. "Click the result" becomes "rendered with textContent, no handler attached — clicking does nothing," with a line number.
  • The contrast is the signal. A wants an outcome; B reports mechanism. Where the two fail to meet is your UX failure.

The setup (≈30 lines of shell)

Each turn is one non-interactive CLI call per agent, threading a shared transcript file:

# B: the app, reading its own code (read-only sandbox, repo mounted)
B=$(codex exec --sandbox read-only -C "$REPO" "$(cat prompt_B.txt)")

# A: the stubborn user (no repo needed — pure persona)
A=$(claude -p "$(cat prompt_A.txt)")

prompt_B.txt"You ARE the app. Read the source. Trace EXACTLY what the user sees after their latest action. Cite file:line. Be brutally honest about dead-ends; never invent behavior the code lacks. TRANSCRIPT: …"

prompt_A.txt"You are <persona> with goal <goal>. React to the app's last response, then keep trying concrete actions. Persist; only stop when truly dead-ended. TRANSCRIPT: …"

Append both turns to the transcript, repeat 5–6 rounds, stop when the user gives up.

A tooling note: for the code-reading agent, use whichever CLI reliably returns one bounded answer per call. For the persona agent, a role-play prompt works on either — just avoid prompts that trip a heavyweight "research" mode, which can background itself and never return a clean turn.

When to reach for it

  • Before a UX pass, to map where intent meets reality.
  • On a flow you think works end-to-end — the disconnect between two subsystems (old button, new pipeline) is exactly what it finds.
  • As a complement to, not a replacement for, real tests. It reasons about code; it does not execute it. Treat its findings as leads to verify, then confirm the real ones with an actual run.

Takeaways

  1. Test that the user reaches the goal, not just that the endpoint returns 200.
  2. Make the "user" agent stubborn — the deep findings live past the first dead-end.
  3. Ground the "app" agent in real code — that is what turns role-play into a bug report instead of fan-fiction.
  4. It is static, cheap, and runs before you have written a single test fixture.

The bug report wrote itself. Now I just had to lob it at my senpais and clock out on time. I have lived humbly, and I intend to keep living humbly — so that this little spell can keep buying me more time to slack off.


The whole rig was a ~30-line shell loop over two CLI coding agents. If folks want it, I'll publish the script as a follow-up.