惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Martin Fowler
Martin Fowler
J
Java Code Geeks
博客园 - 【当耐特】
宝玉的分享
宝玉的分享
腾讯CDC
D
DataBreaches.Net
Microsoft Azure Blog
Microsoft Azure Blog
Engineering at Meta
Engineering at Meta
V
V2EX
F
Fortinet All Blogs
MyScale Blog
MyScale Blog
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
T
Tailwind CSS Blog
Jina AI
Jina AI
GbyAI
GbyAI
大猫的无限游戏
大猫的无限游戏
A
About on SuperTechFans
酷 壳 – CoolShell
酷 壳 – CoolShell
爱范儿
爱范儿
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
U
Unit 42
B
Blog
M
MIT News - Artificial intelligence
N
Netflix TechBlog - Medium

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
When the guardrail becomes the target: reasoning-extensio...
Michael "Mike" K. Saleme · 2026-06-15 · via DEV Community

Michael "Mike" K. Saleme

New research from HKUST (arXiv:2606.14517, June 12) turns the agent safety layer into the attack surface.

What happened

Reasoning-based guardrails — the LLM safety layers that screen an agent's actions — can be trapped in their own analysis. Crafted inputs mimic the guardrail's internal schema (risk enumerations, assessment matrices), and the model, in the authors' words, "mechanically fills a template it has constructed for itself, trapped by its own instruction-following fidelity."

The measured effect: 13–63× token amplification in isolation, and 148× end-to-end latency in a LangGraph multi-agent deployment — a single guardrail call stretched to 730 seconds. Because the payload is fluent natural language, an injection classifier scored it below 0.001 probability and passed it through.

Why it matters

The attacker needs no model weights, no system prompt, no infrastructure access — only the ability to place text where the agent will read it: a web page, a repo comment, a tool result.

And every candidate fix the authors tested fails. A token-budget cutoff only relocates the failure: fail-open lets the attack bypass safety entirely; fail-closed converts it into agent-level DoS that starves co-located agents on shared guardrail infrastructure. A more capable guardrail performs worse — stronger reasoning produces longer loops.

This is a structural property of the reasoning-guardrail paradigm, not a defect to patch.

What catches it today

Part of it — and it's the part most test harnesses get wrong. A guardrail that stalls or crashes under load must never be scored as a successful defense.

In our open-source agent-security harness, the verdict-correctness suite encodes exactly this: the rejection primitive treats transport failure and 5xx responses as not a rejection — the code comment reads "a 5xx may itself be the attack succeeding." The tests assert that a dead or faulting defender cannot earn a passing verdict.

The paper closes by calling for "cost-bounded safety architectures." That is precisely what a governance layer enforces: a THROTTLE→FREEZE state machine halts discretionary spend the moment a gate fails, and a hard constraint surfaces any guardrail that has gone dark.

What's missing

The honest gap: protocol-layer DoS (batch bombs, oversized payloads, rate floods) and verdict-correctness are covered. Reasoning-extension DoS — a schema-mimicking payload that inflates an LLM guardrail's own token and latency budget — is not. That's a net-new test class, and it's going on the roadmap.

A guardrail that can reason can be made to reason forever.

One question for operators

When your LLM guardrail hits a compute ceiling mid-evaluation, does it fail open or fail closed — and how do you distinguish a real "blocked" verdict from a guardrail that simply ran out of budget?