The recent signal
Anthropic engineering leader Fiona Fung, who leads teams behind Claude Code and Cowork, said AI coding agents have changed how her teams work.
The tools help engineers ship more code, but they also make the work lonelier. Developers spend more time working with agents and less time learning directly with other engineers. Fung’s response was to create more intentional collaboration: programming lunches, hackathons, and shared maker time.
That is an interesting culture story.
It is also an engineering story.
Because when developers move from working with humans to operating agents alone, some safety moves out of the room.
Human collaboration used to catch runtime mistakes
In normal engineering work, many mistakes are caught informally.
Someone asks:
Why are we retrying this again?
Why does this script have no timeout?
Why is this touching production?
Why did the dependency change?
Why is the task not making progress?
Why are we calling this API again?
These are not formal verification systems.
They are human runtime checks.
AI coding agents change the workflow.
A developer gives a task.
The agent plans.
The agent edits.
The agent runs tools.
The agent calls models.
The agent retries.
The agent adds context.
The agent tries again.
That loop can be useful.
It can also continue long after a human teammate would have stopped the run.
Agents are loops
A basic model call is simple.
You send input.
You get output.
You inspect the result.
An agent is different.
An agent can run through many steps:
while (!taskDone) {
const plan = await model.call(context);
const toolResult = await runTool(plan.tool);
context = updateContext(context, toolResult);
}
That shape is powerful, but incomplete.
There is no budget.
No max-step limit.
No retry-storm detection.
No prompt-loop detection.
No unknown-pricing check.
No no-progress detection.
A safer version has to ask whether the next provider call should happen at all.
const decision = guard.beforeCall({
runId,
model,
prompt,
stepCount,
budget,
previousPrompts,
retryState,
});
if (!decision.allowed) {
throw decision.error;
}
const result = await provider.call({
model,
prompt,
});
The important part is placement.
The guard runs before the provider call.
Not after.
Why dashboards are not enough
Dashboards are useful.
They show usage.
They show cost.
They help debug.
But they are usually post-execution.
They answer:
What happened?
Agent runtimes need to answer:
Should this next call happen?
That question matters because most AI-agent cost failures are not one huge request.
They are usually sequences of normal-looking calls.
A retry.
A similar prompt.
A tool call.
Another retry.
A longer context.
Another model call.
Each step may look reasonable.
The whole run may be waste.
Useful pre-call checks
A practical agent runtime should stop execution when basic safety conditions fail.
Budget guard
A run should have a task-level budget.
If the next estimated call crosses that budget, stop before the provider call.
Max-step protection
An agent that cannot finish within a reasonable number of steps may be stuck.
Max-step limits are not fancy.
They are production safety.
Retry-storm detection
Retries are normal.
Retry storms are not.
The runtime should detect when the agent keeps repeating failing behavior.
Similar prompt-loop detection
If prompts are nearly identical across repeated attempts, the agent may not be exploring new information.
That should trigger a stop or structured failure.
Unknown model pricing
If the runtime does not know the price of the model, it should fail closed.
Guessing inside an autonomous loop is dangerous.
Where AI CostGuard fits
This is the layer I am building with AI CostGuard.
AI CostGuard is a local-first TypeScript runtime safety layer for AI agents.
It is designed to prevent expensive agent failure modes before provider calls execute:
retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior
It is developer-focused.
No SaaS by default.
No cloud dashboard by default.
No login.
No tracking.
The goal is not to replace provider dashboards.
The goal is to stop known bad runtime patterns before they become provider spend.
The takeaway
AI agents make individual developers more capable.
They also reduce the human friction between intention and execution.
That means some safety has to move closer to the runtime.
If an agent can run alone, it needs limits.
If it can retry, it needs retry control.
If it can loop, it needs loop detection.
If it can call providers, it needs budgets.
If it can run unattended, it needs a kill switch.
https://github.com/salimassili62-afk/ai-costguard