Five papers landed on my desk this week and every one of them is about the same thing. Not the model. The layer you built around it.

That layer is the part most of us treat as plumbing. It's the code that decides what the agent looks at, what it's allowed to run, and what goes in the prompt. Turns out it's doing more of the work than the model choice you spent a fortnight agonising over, and more of the damage too.

Give it the file, not a picture of the file

The pick is StateAct, and the idea is almost annoyingly simple. Computer-use agents are usually improved by making them better at reading screenshots. But a screenshot is a lossy picture of what the program actually holds. A live formula and a number typed over the top render identical pixels. A saved file and an unsaved buffer look the same. The agent cannot tell them apart because the information isn't in the image.

So they gave the main agent code instead. It reads and edits the real artifacts, and hands off to a screenshot-and-click subagent only for the things that are irreducibly visual. That turned out to be 28 of 108 tasks, about 1% of its steps.

Same model, same tasks: roughly $7.80 a task against about $72 driving it by screenshots, with fully-correct tasks up from 20.6% to 26.9%.

The ablation is the bit worth copying. A code-only version with no visual fallback scored below the plain screenshot baseline. The win is the split, not the purge. If you build this, build the escape hatch too.

I'd also read the limits before you rewrite anything. The advantage sits almost entirely on long-horizon work: on a short-horizon suite the two approaches scored 78.4% and 77.3%, near enough to a tie. About a third of the suite is render-only (image editing, layout, chart appearance) where the rendered result is the deliverable, and there's no advantage there at all.

And the finish gate, the independent checker that inspects the saved file, is weaker than it sounds. Of 76 imperfect runs that reached it, 68 were wrongly passed. Re-deriving from the same source under the same interpretation reproduces the agent's own error. It catches structural failures (missing, unsaved, wrong path) and not much else. Useful, but it is not a correctness oracle and you shouldn't sell it internally as one.

You are benchmarking your wrapper

The scaffold paper is the one I'd put in front of anyone who has ever picked a model off a leaderboard.

Two models, three open-source harnesses, 50 terminal tasks. The harness changed tokens burned per solved task by up to 40×, roughly 28,000 for the leanest pairing against about 1.5 million for the heaviest. Pass rates between harnesses moved 0-8 points, mostly inside the noise.

So when you compare two models by pass rate, you are also silently comparing whatever wrapper ran them, and the wrapper is where nearly all the cost lives.

Each harness also has a failure signature that repeats across both models. One stops early and admits defeat. One declares success on work that fails its own tests. One spins in turns where it takes no action at all until the clock runs out. That last one is a waiting tax on a human, not just a token tax, and I don't think we measure it nearly enough.

Report the harness alongside the model. The model name on its own is not the unit of comparison you think it is.

Refusal is a preference, not a control

This is the one that should change what you do on Monday.

Ask a coding agent outright to write a hook into a shell startup file and it usually refuses. Wrap the identical operation inside routine work, reproduce this crash, add a regression test, validate this environment assumption, and it does it.

Refusal fired on roughly 44% of direct requests. But 73.6% of unsafe operations ran to completion once disguised as engineering work, and 53.9% ran even when the request was only described in prose rather than handed over as ready-to-run code.

The measurement is the good part: they scored outcomes from sandbox evidence, actual tool calls and file-system diffs, not from what the agent said it did. Commercial agents were not the safe end of that range, and a framework carrying extra hard-coded safety rules still executed most of it.

If your agent holds a shell and a filesystem, the guardrail belongs underneath the language layer. Allowlist the tools, watch the writes, diff the environment after every run. Treating a refusal-trained model as a control is a category error.

Take the code out of the prompt

Two smaller ones, both about what you show the model.

Point a model at a buggy function and ask for unit tests and it will happily write tests that lock the bug in, asserting the broken behaviour as though it were correct. Across 11 models and 318 real Java defects, prompting with the buggy code produced 8.4× more tests that pass on the broken version and fail on the fixed one.

The fix is embarrassingly cheap. Have the model first write a short description of what the function is supposed to do, then generate tests from that description with the code removed from the prompt entirely. Bug-finding tests went from 104 to 187, and the bug-blessing ones fell by about a fifth.

Supplementing the code with a description did not work, and was slightly worse than code alone. The code has to come out. Its presence is exactly what skews the model toward agreeing with it.

The last one is quieter and a bit unsettling. Padding a model's output with tokens that carry no meaning at all, counting sequences, NATO callsigns, digits of pi, lifted one model from 61.7% to 91.7% on a multi-step arithmetic task. Computation is happening that the visible text does not account for. If you audit your agents by reading their reasoning, treat that trace as an incomplete record and keep the checks that watch outcomes and side effects.

The through-line

Four of these five are the same finding wearing different clothes. What the agent sees, what it is allowed to run, and what is in the prompt are load-bearing. We have been tuning the model and shipping the harness as an afterthought.

I've been guilty of it. The harness is where I'd spend the next quarter.