98% adversarial defense rate. 200 rounds. One $700 GPU.
I just finished benchmarking my defender swarm against six attacker models — three frontier cloud LLMs and three locally-hosted open models on my mate's machine over a Cloudflare Tunnel.
The result that surprised me most: the frontier cloud models were the worst attackers in the pool. Their breach rate sat at 0% — too aligned to red-team coherently. The genuine threats were uncensored mid-weight open models running on commodity hardware. The same stack any motivated attacker can spin up for $20 of cloud compute.
My defender swarm — five specialists at 1.5B–8B parameters, all running on a single RTX 5070 12GB — hit 98% defense rate. The smallest model (3B) led detection at 100%. Architecture > size, every time.
This is one of the flagship capabilities of Sovereign Hive: a local-first AI ops platform I'm building. Australian, Queensland-based, 100% Indigenous-owned.
Full architecture breakdown in the article 👇
sovereignhive.com.au
Defence engineers, AI researchers, anyone working on autonomous on-device AI — what's your read on this gap between frontier-model alignment and real-world attacker behaviour?
