How to Keep AI Coding Agents from Hallucinating: A Guide to Harness Engineering
AI coding agents (like Claude Code, Devin, or open-source equivalents like OpenClaw) are incredibly powerful. They can navigate directories, write tests, refactor modules, and submit PRs.
Yet, if you drop them into a raw repository without boundaries, they suffer from context window pollution, agent amnesia, and scope drift. A simple bug-fix refactor can trigger a 6-hour loop where the agent rewires half the project, deletes unrelated tests, and gets stuck in "process theater."
To fix this, we need Harness Engineering.
What is Harness Engineering?
An Agent Harness is a structured, repository-local control layer designed to guide and verify the agent's work. Instead of feeding your LLM a monolithic prompt, you embed a lightweight system of record and physical feedback loops directly inside the workspace.
I have packaged the exact, battle-tested Markdown-based context rules I use to steer and constraint my local agents into a public repository: MasihMoafi/harnesses-I-use.
Rather than complex code, this repo shares raw configuration rule sheets:
-
AGENTS.md: The global router and primary entry point. Every AI agent reads this first to understand the Priority Kernel (reply constraints, English defaults, local-first behavior). -
CODEX_CODING_GUIDELINES.md: Surgical coding rules adapted from Andrej Karpathy's coding practices. Enforces minimal changes, style-matching, and goal-driven execution. -
TERMINAL_AND_GIT_RULES.md: Safe terminal navigation, SOCKS/HTTP proxy configuration, explicit git scoping (e.g., never usinggit add -A), and change safety (using Ubuntupkexecfor root commands instead of raw CLI password prompts). -
SESSION_HANDOFF_RULES.md: Guidelines for context awareness and continuity. Defines how state should be captured and transferred between sessions. -
ARTIFACT_RULES.md: Rules for generating verified visual and math outputs, including local HTML layout verification, MathML for formula rendering, and PDF inspection. -
abbn.md: A dictionary of standard abbreviations (e.g.,ctu= continue,fmy= familiarize,ver= verify) to save token count and maintain short, high-efficiency communication.
The Karpathy Connection: Simplicity & Rigor
This harness approach is heavily inspired by Andrej Karpathy's open-source education repos (like micrograd and makemore).
Karpathy’s projects are celebrated because they strip away bloat. They focus on clear, reproducible mathematical baselines and avoid over-engineering.
We applied that same philosophy to agent-driven code generation. The core rules of our harness require:
- Surgical Changes: Touch only the exact lines requested. No drive-by style refactoring of adjacent code.
- Verifiable Outcomes: Before the agent claims a feature is complete, it must run a local script to verify the baseline, compile the code, and attach raw terminal output logs to its session record as physical evidence of success.
- Harness Simplification: Every rule is considered overhead. As models grow smarter, we run tests to prune redundant rules.
Case Study: Autonomously Generating ML Research Reports
To test this, I used this exact harness to build a comparative machine learning research project: Sensor Fault Diagnosis.
The agent was given a realistic synthetic sensor dataset and tasked with:
- Training a Bayesian classifier (Gaussian Naive Bayes) and mapping conditional probabilities.
- Building and training a PyTorch MLP neural network.
- Calculating and comparing multiple losses—including standard Binary Cross-Entropy (BCE), micrograd-style Max-Margin (Hinge) loss, and makemore-style Cross-Entropy loss.
- Exporting ROC curves, decision boundaries, and generating a publication-grade LaTeX/HTML research paper.
By restricting the agent to a single control surface (a structured manual) and enforcing strict keep/discard criteria, the agent completed the pipeline and wrote the final report entirely autonomously.
Without a harness, the agent would have bloated the repository with decorative dashboard scripts or fake performance metrics. The harness kept it grounded.
Get the Harness
If you are building code with AI agents, stop writing 2,000-word system prompts. Start building repository harnesses.
Check out the templates and configurations:
👉 MasihMoafi/harnesses-I-use
For more of my work, experiments, and research, check out my website:
👉 masihmoafi.tech