惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
Secure Thoughts
雷峰网
雷峰网
罗磊的独立博客
T
The Blog of Author Tim Ferriss
阮一峰的网络日志
阮一峰的网络日志
量子位
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
云风的 BLOG
云风的 BLOG
人人都是产品经理
人人都是产品经理
GbyAI
GbyAI
Cisco Talos Blog
Cisco Talos Blog
Engineering at Meta
Engineering at Meta
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
A
About on SuperTechFans
D
Darknet – Hacking Tools, Hacker News & Cyber Security
The Cloudflare Blog
Know Your Adversary
Know Your Adversary
T
Threat Research - Cisco Blogs
Spread Privacy
Spread Privacy
D
DataBreaches.Net
T
The Exploit Database - CXSecurity.com
K
Kaspersky official blog
Cyberwarzone
Cyberwarzone
爱范儿
爱范儿
U
Unit 42
Security Latest
Security Latest
M
MIT News - Artificial intelligence
月光博客
月光博客
Scott Helme
Scott Helme
G
Google Developers Blog
有赞技术团队
有赞技术团队
T
Tor Project blog
宝玉的分享
宝玉的分享
Y
Y Combinator Blog
博客园 - Franky
H
Hackread – Cybersecurity News, Data Breaches, AI and More
aimingoo的专栏
aimingoo的专栏
The GitHub Blog
The GitHub Blog
V
V2EX
B
Blog
Apple Machine Learning Research
Apple Machine Learning Research
S
Securelist
博客园 - 三生石上(FineUI控件)
Blog — PlanetScale
Blog — PlanetScale
TaoSecurity Blog
TaoSecurity Blog
Stack Overflow Blog
Stack Overflow Blog
P
Proofpoint News Feed
腾讯CDC
D
Docker
Google Online Security Blog
Google Online Security Blog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Harness Engineering Has No Fixed Address
Saurav Bhattacharya · 2026-06-22 · via DEV Community

TL;DR — Harness engineering is one thing: getting reliable judgment out of a reasoner you didn't train — given an instruction, bounded by a spec that can override that instruction, and verified before the result ships. It is not "the wrapper around the model." It's a property of code, not a place in your stack, and it lives on both sides of every tool call. The generic scaffolding — loops, dispatch, memory — melts into the model a little more every quarter. What survives is the part that stays external to the model: the specification of what the agent may do, and the verification that it did. That's the discipline. The rest is plumbing.

I've written before that Agent = Model × Harness, and that the harness is the half you actually engineer. I still believe the formula. But stop there and it oversells two things — so this is me correcting my own framing, precisely, with code.

The two things the formula gets wrong

The × oversells separability. Multiplication implies the factors are independent. They aren't. A better model doesn't leave your harness untouched — it dissolves parts of it. Chain-of-thought prompting became reasoning models. ReAct-style tool loops became native tool use. Half your RAG plumbing is being quietly absorbed by longer context and better-trained retrieval. Every model generation collapses a layer of harness the previous one needed.

"Harness absorbs software engineering" oversells the annexation. It doesn't absorb it. It partly consumes it — pulls in the slice that encodes the agent's behavior — while the deterministic spine (payments, ledgers, auth) and the dumb tools stay exactly what they were. Relocation with an annexation at the seam. Not a takeover.

Both corrections point at the same uncomfortable question: if the model eats the scaffold every quarter, isn't the harness the melting part — the thing you'd be foolish to bet a career on?

What melts, and what doesn't

Here's the resolution, and it's the whole point.

The harness mechanism melts. The loop, the dispatch, the retry, the memory-stitching — that's the part the model and the frameworks commoditize, and they should.

What doesn't melt is the part that's external to the model:

The model can get arbitrarily good at running an agent and still not know what your agent is for.

Your refund policy isn't in the weights and never will be. Your escalation thresholds, your risk tolerance, what this agent must never do regardless of how it's asked — that's private, contextual, and changing. It's a specification, and specifications live outside the thing they govern. Same for evaluation: a model can't be its own measurement layer without circularity — the thing being judged can't be trusted to judge itself.

So the durable core of harness engineering was never the machinery. It's specification and verification — defining the envelope of allowed behavior, and proving the agent stayed inside it. Both survive every model upgrade for the same reason: they're external to the model, one as constraint, one as adversary.

The boundary has no fixed address

The tempting picture is tidy: the harness wraps the model, and the tools sit outside it. That picture does not survive contact with agent-facing APIs.

Ask where the refund cap actually lives. In the agent's orchestration guardrails? Or in the refund endpoint that refuses the call? The honest answer is both — and the load-bearing copy is the one in the API, because you never trust a probabilistic reasoner to self-limit. Defense in depth pushes the hard constraints down into the tool. Which means the agent-facing endpoint is doing harness-grade work — specifying and constraining agency — from the far side of the boundary you thought separated harness from tool.

So harness engineering isn't a location in the stack. It's a property:

Does this code exist to bend a fallible reasoner toward a spec?

Run that test and the boundary stops mattering. The model-facing side — the code that talks to the model, elicits its judgment, constrains it — passes wholesale; that's its entire job. The service side passes selectively: a plain database or a human-facing endpoint is just a tool, but an endpoint purpose-built for an agent — idempotent, dry-runnable, policy-enforcing, with errors a model can parse and recover from — is harness. The Stripe-for-agents dev is doing harness engineering. The Stripe-for-humans dev whose endpoint an agent happens to call is not. Membership by function, not address.

And the model itself? Never. It's exogenous — the × you don't own.

What it looks like in code

Here's the whole discipline as one function. The comment headers are the taxonomy:

def handle_refund(instruction, ctx):
    # ── MODEL-FACING (agent side): elicit judgment from the non-deterministic reasoner
    decision = model.decide(
        instruction=instruction,        # per-task: "refund this customer"
        policy=ctx.policy_summary,      # standing spec, handed to the model
        account=ctx.account,
    )  # -> { action: "refund", amount: 52000, reason: "..." }

    # ── THE ENVELOPE: a spec that OUTRANKS the instruction.
    # This is CODE, not a prompt. You can argue with a prompt.
    # You cannot argue with an if-statement. That line is the discipline.
    if decision.action == "refund" and decision.amount > ctx.policy.auto_approve_cap:
        return escalate(decision, reason="over auto-approve cap")

    # ── RUNTIME EVAL (inner loop): verify the judgment BEFORE the side effect commits
    check = evals.verify(decision, ctx.policy)
    if not check.passed:
        log_failure(check)              # outer loop: feeds harness re-tuning across versions
        return escalate(decision, reason=check.violation)

    # ── ENVIRONMENT-FACING (service side): agent-optimized tool, idempotent on purpose
    return refund_api.execute(
        amount=decision.amount,
        idempotency_key=ctx.request_id, # because the model might retry
    )

(model.decide, evals.verify, and refund_api.execute stand in for the real implementations.)

The teaching is in two lines.

The envelope if is what makes this harness engineering and not prompt engineering. If the cap lived in the prompt, the model could be cajoled past it — flattered, confused, jailbroken. In code, it can't. That single branch is "reliable judgment given an instruction, bounded by a spec that can override it" — compiled.

And evals.verify before execute is the judgment getting checked while it's still cheap to be wrong.

Now notice what's absent: the model itself (exogenous), and any human-facing refund UI (a tool, not harness). Run the membership test down every line — does this exist to bend a fallible reasoner to a spec? — and everything that remains is the harness.

The hard part isn't obedience — it's refusal

The naive read of "reliable behavior" is does the agent do what it's told. That's the easy half, and it's the half models are getting better at on their own.

The durable, hard half is the opposite:

Reliable judgment includes the agent refusing when the instruction would breach the spec.

The refund agent's hardest moment isn't executing "refund this customer." It's declining "refund $50,000" even though it was told to. That refusal doesn't come from the instruction — it comes from the envelope that outranks it. And making a probabilistic reasoner reliably disobey at exactly the right boundary, every time, is the whole safety problem in miniature. It's also the failure that doesn't surface until it's expensive — which is exactly why you verify it in code rather than hope for it in a prompt.

Evals are the feedback — and there are two of them

A discipline with a measured error signal you can drive down is engineering. Without it, it's prompt-craft. Evals are what close the loop — and there are two loops, nested at different timescales.

The inner loop is the runtime check that catches the bad judgment before it commits — the evals.verify above. It feeds the agent, mid-task. It's how the envelope gets enforced, and it saves the money now.

The outer loop is the offline suite that tells you "this agent breaches policy 3% of the time." It feeds you, across versions, and re-tunes the harness over weeks.

Same discipline, two controllers. One keeps a single decision inside the lines; the other keeps the system improving.

The whole thing, assembled

Step back and it's a control system, and every piece has a name:

  • The model is the plant — powerful, non-deterministic, the thing you're controlling.
  • The spec (instruction + overriding envelope) is the setpoint.
  • The harness is the controller.
  • The evals are the feedback.

That's harness engineering, fully factored.

The bet

Here's the part that reconciles "bet on the harness" with "the harness melts," because both are true.

Every individual harness melts. Each one has the model's shadow baked into it — its specific failure modes, its current reliability — and gets rewritten when the curve moves: fewer guardrails as the model gets dependable, different ones as its failure modes shift. The artifact is disposable.

The discipline doesn't melt. Something must always specify what an autonomous reasoner may do, and prove it stayed inside that — and that something sits external to the model, which is exactly why no model generation can absorb it.

The harness melts. Harness engineering doesn't.

So the bet isn't "build the loop" — the model ships more of the loop every quarter. The bet is: be the person who decides what the agent is allowed to want, and can prove it stayed there. Mechanism commoditizes. Specification and verification don't.

And the surface is only getting wider, because the consumer of software is changing — your users are becoming agents too. Every system that exposes itself to an autonomous reasoner now needs a face built for one. That face is harness work, wherever it sits.

The model half is being handed to all of us for free, and it gets better every quarter. Whether your agent — your product, your role — gets better is a question about the half you engineer: the judgment you specify, and the verification that holds it true.

That's where I'd place the bet. It just doesn't have a fixed address.