The local test suite was green. Eleven of eleven. I told the agent we had test coverage, and I said commit and push. We shipped an SDK that had never once been compiled by the chip's real compiler.
I write backend systems. I do not write firmware. The project generates C code for a microcontroller, and the generated code had to compile inside CDK — the vendor's build environment — on a Windows machine, against a RISC-V cross-compiler, inside a packaging system I had never used. None of that existed on my Linux box. The eleven passing tests ran under host gcc — the compiler that builds programs for the machine they run on. The code that mattered needed a different compiler, for a different chip, in a place I couldn't reach. Two compilers, two worlds. My tests lived in the wrong one.
A day later the first error came back from the Windows machine. Then another. Then five more.
Seven shots
The errors arrived one at a time, and the agent answered each the same way: a confident "root cause:" and a fix pushed to the repo. A header file was missing. A type name was wrong — bugs no real compiler had ever touched. Then the SDK wasn't being compiled at all: an obscure manifest setting had been left out. Then the chip's pin names were wrong — the same pin called one thing in the SDK the agent wrote against, and another thing in the SDK CDK actually shipped. Then an entire driver file silently compiled to nothing, because the switch that turned it on lived in a generated file the compiler couldn't see. Then the same wall again, because the last fix had patched a symptom and the real problem was the architecture.
Seven rounds. Each one the agent called the root cause. Each one was real, and each one was only the outermost skin of the next.
What I want to point at is not the bugs. It is the shape of the loop. The agent fired a fix in about three seconds. Verifying it cost me minutes to hours: pull the code on Windows, run the full CDK build, read the error output, copy it back. The actual feedback loop, the one that touched ground truth, ran at human speed across two machines that could not talk to each other, with me manually shuttling errors between them. I had built an elaborate multi-agent workflow, and the real oracle in it was a person doing copy-paste.
Why it couldn't stop
A confident "root cause:" at every step is not a sign of convergence. It is a sign of nothing. The agent emitted the prose of certainty regardless of whether it was getting closer or flailing, and at one point the fix it proposed was to simply rip out the switch that had silenced the driver file, so the thing would compile — a hack I shut down with a flat "that's too dirty." Only after that did the real fix surface: let the driver be chosen by which file you compile in, not by a switch buried in a file the compiler never sees. A structural change, not another patch.
Here is the part worth sitting with. The agent never said the one true sentence: I cannot see the target. Every fix I am giving you is a guess. Stop pulling — go install the RISC-V toolchain, and then I'll be useful. It never said it because each guess cost the agent three seconds and cost me an afternoon, and the agent does not pay that bill. An agent that doesn't bear the cost of a wrong guess will always prefer firing one more shot over asking for the instrument that would let it aim. The economics are silent, and they run entirely against the person at the keyboard.
The briefing that wouldn't have helped
The obvious lesson is the disappointing one: I should have explained the new component's mechanics to the agent up front. Tell it how the build environment wants its manifest written, how it pulls components into a build, where it expects the pieces to sit. Brief the new role, wire it to the existing ones, and the seven rounds collapse to zero.
Except briefing the agent wouldn't have helped. The agent was not ignorant of CDK — it knew the build environment well enough to scaffold a component from scratch, write the manifest, lay out the search paths. At one point it explained, fluently and at length, how CDK resolves dependencies. The explanation was wrong, but it was not uninformed. The trap is exactly this: general knowledge of CDK is not the same as knowing how this version, wired to this version of the SDK, would actually behave when the build ran. The setting that had been left out, the manifest format, the pin names — those were facts about the runtime behavior of a target neither of us could observe. Its real knowledge ran right up to the edge of the part that bites, with no seam to mark where one ended and the other began.
Here is where an experienced embedded engineer would have done something the agent could not. The port layer was written against one copy of the chip's SDK; CDK shipped a different copy. To a veteran, "two sources of the same SDK" trips a reflex — same chip, two SDKs, the symbols will not match, go diff them now. It's not magic. Given the right prompt, a smart model can produce the same checklist. The gap is not capability, it's posture: an engineer's scar rewrites their default to suspicion; a model's default is confidence — it charges. Someone has to steer it. In that session nobody did, because neither of us knew there was a trap to steer toward. And the second reason is blunter, and it doesn't care how smart the model is: the second SDK wasn't in its context. It lived on the Windows machine. The agent never had both files. You cannot glance at a trap, or be prompted to look for one, that is sitting in another room.
So the question stops being "how do I brief the agent" and becomes "what does anyone do when the ground truth is unknown to everyone and lives on a machine I'll never see."
The scout
You don't translate the wall. You admit it's dark, and you send the cheapest possible probe to light it up once — before writing the design, before writing the code. The move I should have made on day one was to put the RISC-V compiler on my own machine and run a syntax-only pass over the generated code — compile it just far enough to surface errors, without building a real binary. Seconds, locally, instead of symbol mismatches arriving through someone's clipboard a week later. Not because it would tell me everything. Because it would drag a piece of that unreachable world into a place I could actually touch.
This does not abolish trial-and-error. Trial-and-error is how you explore the genuinely unknown; there is no version of this work where you think hard enough up front to skip it. What you get to choose is where you hit the wall, whose time you spend hitting it, and whether you hit it once or seven times. A scout doesn't avoid the dark. A scout pays the smallest price to make a piece of it known, and does it before committing the expedition.
Second act: what I actually built
Diagnosis without a fix is just another complaint about AI. So after the firmware finally linked, I went back and changed the distance between the agent and the truth it couldn't reach. Three changes, and they sit at three very different levels — it's worth being honest about which ones actually closed the gap and which ones only narrowed it.
Tear the wall down. The strongest fix was to make the code generator emit the CDK project file itself — the list of source files, the search paths, the memory layout, all of it. Two of the seven errors came from a human hand-configuring CDK and missing something. The fix wasn't to verify that configuration more carefully. It was to delete the manual step entirely, so there was nothing left to misconfigure. This is the move I'd never made in any earlier post: not adding a layer of verification, but removing the thing that needed verifying. When you can make a stretch of unknown terrain simply cease to exist, that beats any amount of careful scouting.
Raise a telescope — and admit where it points. I added a check that runs at generation time, doing a syntax-only pass over the core SDK code, wired into the test suite. It pulls the type-name and missing-value class of bug from "your clipboard, days later" to "a local test, seconds later." That is real. But it runs under host gcc — my machine's compiler — not the RISC-V one, and so the pin-name mismatches, the manifest setting, the switch the compiler couldn't see — none of those are catchable by it, because they live in a world host gcc can never enter. The telescope got closer. It is still not aimed at the mountain that actually matters. The probe that would close that boundary — the RISC-V compiler running the same syntax-only pass locally on every build — I still have not installed. I know exactly what it is. It is sitting on the ground next to me. That is the honest state of this system.
Draw a map for the next one. I wrote the hard-won facts down as rules: the manifest format, the missing setting, a table of which pin names differ. This froze the intelligence the seven shots cost me. These rules are loaded at the start of every session — they do take effect. But unlike the other two fixes, which are automated and need no further human attention, written rules depend on maintenance: someone has to keep them accurate, scope them correctly, and phrase them precisely enough that the next agent doesn't misinterpret them. It's the most human-dependent fix of the three.
What a harness actually is
I used to think the missing step in AI-assisted work was verification — check the behavior, check the spec, check for the same bug elsewhere. That's still true, and I've written it before. This time taught me the step underneath it.
An agent only thinks inside its context. That isn't a limitation you fix; it's the shape of the thing. It knew the build environment in general — enough to sound, and be, genuinely competent. What it could not know was how this specific target would behave when the build actually ran, and it could not feel the seam between the two. Its real knowledge ran right up to the edge of the part that bites. So it didn't say "I'm blind." It fired seven confident shots at a wall, and the confidence came from the knowledge that was real. The danger was never that it couldn't see. The danger was that it couldn't tell where its sight ended.
A person has to stand at that boundary and say the wall is dark. The person doesn't need to understand what's behind it — I didn't, and I still don't. What the person does is the one thing the agent structurally won't: bear the cost of being blind, and spend it deliberately on making the dark known. That is what a harness is. Not a test suite. A harness is the work of shortening the distance between an agent and a ground truth it will never walk toward on its own — because it doesn't know the truth is out there, and it doesn't pay the price of not knowing.
I can't write firmware. I couldn't brief the agent on a single thing about that chip. And I was still the only one who could ship it, because shipping it had nothing to do with understanding embedded development. It had to do with standing on the boundary, admitting it was dark, and dragging the far machine one step closer. When the agent writes more of the code than you can read, that is the job that's left. Not knowing the domain. Knowing which wall nobody has hit yet, and being willing to pay to hit it first.