Two Models Just Hit 90% on Agent Coding. One Cost Less Than a Penny.

By Vilius Vystartas | May 2026

Ten more models through the same 10 agent coding tasks. Two tied the all-time record. One cost $0.0002. The other hit the score at $0.0018 — cheaper than most models scoring 70%.

Batch 10 was the cheapest one yet.

The Leaders

Two models scored 90% with zero hard fails, joining MiniMax M2 Her and Baidu Ernie 4.5 300B as the highest-scoring models on this benchmark:

Qwen3 Coder 30B A3B — 90% in 28 seconds, $0.0004. An efficient coder that doesn't burn budget on thinking tokens it doesn't need.

DeepSeek Chat (original) — 90% in 59 seconds, $0.0018. The original DeepSeek Chat still competes with modern models on agent coding. Newer doesn't always mean better.

The Surprises

LFM 2 24B A2B (85%, $0.0002, 15s) is the cheapest model I've ever tested. Liquid's debut family is absurdly cost-effective. A full 10-task benchmark for literally $0.0002. At this price/performance ratio, there's no excuse not to test a model before committing to a more expensive alternative.

Mistral Small 3.2 (85%, $0.0004) is a clear upgrade. The Small line went 75% → 85% across versions — a ten-point jump at the same budget tier. Mistral keeps improving the right things.

Qwen3 14B scored 0% across all 10 tasks. Mandatory thinking mode that can't be suppressed at 300 tokens means every request times out before producing output. Skip for agent coding.

Cydonia 24B V4.1 (80%, $0.001) debuts a new family from TheDrummer. Zero hard fails. Watch this one.

The Duds

Qwen3.7 Max (85%, $0.13, 295 seconds) scored the same as budget models costing 300x less. Thinking mode tax at work — the accuracy is there, but you'll wait five minutes and pay for every second.

Claude Opus 4 (80%, $0.10, 76s) had one hard fail. For a top-tier premium model at $0.10 per 10 tasks, that's below expectations. It's not a bad model — it's overkill for agent coding at a tight token budget.

Aion 1.0 (80%) had two hard fails and was the slowest at 160 seconds. The architecture is interesting, but it's not ready for production agent work.

My Picks

Best overall: Qwen3 Coder 30B A3B — 90%, 28s, $0.0004
Best value: LFM 2 24B A2B — 85%, $0.0002 total. Ridiculous price/performance.
Fastest: LFM 2 24B A2B — 15 seconds flat
Most improved: Mistral Small 3.2 — 75% → 85% across versions
Skip entirely: Qwen3 14B for agent tasks

Methodology

Ten real-world agent coding tasks — file operations, shell commands, error recovery, data parsing — tested against each model via OpenRouter. Max tokens: 300. Temperature: 0.1. Results scored by pattern matching against expected outputs. Pre-flight verification caught 2 models (Ernie 4.5 21B — HTTP 429, Trinity Mini — empty content) before they wasted the batch.

Total batch cost: $0.14 across 9 models. Qwen3.7 Max alone accounted for $0.13 of that — thinking tax.

Total models tested: 148 (up from 138).

Full results and per-task scores: benchmarks.workswithagents.dev

Because you should.

推荐订阅源