惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

I
InfoQ
S
SegmentFault 最新的问题
N
Netflix TechBlog - Medium
B
Blog
Jina AI
Jina AI
人人都是产品经理
人人都是产品经理
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
H
Hackread – Cybersecurity News, Data Breaches, AI and More
博客园 - 聂微东
Last Week in AI
Last Week in AI
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
V
V2EX
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
大猫的无限游戏
大猫的无限游戏
U
Unit 42
J
Java Code Geeks
IT之家
IT之家
aimingoo的专栏
aimingoo的专栏
博客园 - 叶小钗
T
The Blog of Author Tim Ferriss
博客园 - 【当耐特】
Hugging Face - Blog
Hugging Face - Blog
WordPress大学
WordPress大学
腾讯CDC

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Two Knowledge Hierarchies: Structuring Context for AI Age...
Oscar Rieken · 2026-05-27 · via DEV Community

TestSmith has two distinct audiences that need context about the project: AI agents that work on the TestSmith codebase (helping develop and extend it), and the LLM that generates test code for your project at runtime. These are different problems with different solutions.

Layer 1: Agent Context — CLAUDE.md Hierarchies

When an AI agent opens TestSmith to fix a bug or add a feature, it needs to understand the codebase structure without reading every file. A single large context file doesn't work well — an agent fixing a retry bug doesn't need to know the Java driver's fixture generation logic.

The solution is a CLAUDE.md hierarchy:

CLAUDE.md                              ← package map, invariants, dependency direction
internal/domain/CLAUDE.md             ← interfaces, key types, "add a field" checklist
internal/generation/CLAUDE.md         ← pipeline data flow, verifier selection
internal/llm/CLAUDE.md                ← middleware stack, batch vs fan-out, cache key
internal/projectknowledge/CLAUDE.md   ← TESTSMITH.md hierarchy, budget tiers
internal/drivers/CLAUDE.md            ← how to add an adapter or language driver

Enter fullscreen mode Exit fullscreen mode

The root file is the map. The per-package files are the territory. An agent touching the LLM retry logic loads internal/llm/CLAUDE.md — it never sees the driver or generation docs.

The root file contains three things that every agent needs regardless of task:

  1. Package map — what each internal package does and which files to read first
  2. Dependency direction — the hard architectural constraint (domain never imports other internal packages; drivers never import generation)
  3. Invariants — things that must remain true across all changes (e.g., GeneratedFile.Language must always be set; resolveAction has specific rules for fixture vs. non-fixture files)

Per-package files contain the "read this before touching this package" context: data flow diagrams for the pipeline, the middleware stack for the LLM layer, the adapter registration pattern for drivers.

When Claude Code loads a file in a package, it automatically reads that package's CLAUDE.md. The agent gets exactly what it needs, nothing more.

Layer 2: Runtime LLM Context — TESTSMITH.md

This is what TestSmith injects into prompts when generating tests for your project. It's a conventions file you maintain alongside your source code.

Two levels are merged at generation time:

<project-root>/TESTSMITH.md     ← always loaded; project-wide framework, mock style
<source-dir>/TESTSMITH.md       ← optional; package-level overrides

Enter fullscreen mode Exit fullscreen mode

Example root TESTSMITH.md:

# Project conventions

Framework: pytest
Mock style: pytest-mock (use `mocker.patch`, not `unittest.mock.patch`)
Assertion style: plain assert statements

# Module structure
Services are in `src/services/`. Each service has a single public class.
Tests go in `tests/` mirroring the `src/` structure.

Enter fullscreen mode Exit fullscreen mode

Example per-directory override in src/services/payment/TESTSMITH.md:

# Payment service conventions
This module integrates with Stripe. Mock all `stripe.*` calls.
Use `pytest.mark.vcr` for HTTP interaction tests.

Enter fullscreen mode Exit fullscreen mode

The root file is loaded once at startup and cached in ProjectContext. The per-directory file is merged lazily — only when a file in that directory is being generated. A large monorepo never loads context it doesn't need.

Both files go into the system prompt, not the user prompt. This matters because the user prompt is subject to a configurable token budget (PromptTokenBudget, default 6,000 tokens) with a priority-based trim:

Priority Content Dropped when?
1 (never) Source code Never
2 Internal dep signatures Budget exceeded after source
3 Style snippet from nearby tests Dropped first

Project knowledge is exempt from this budget entirely — it stays in the system prompt regardless of how large the source file is.

Dynamically Mined Conventions

Beyond TESTSMITH.md, TestSmith also mines conventions from existing tests in the same directory — up to 5 files, capped at 80 lines total. This gives the model real examples of the project's test style without requiring the developer to maintain a conventions doc.

This is cheaper and more accurate than a hand-written guide: it automatically reflects the actual test patterns in use, and it updates itself as tests evolve. If your team starts using a new assertion pattern, the next generation run picks it up.

The Dependency Signature Index

The third piece is the dep index: at the start of a --all run, TestSmith analyses every source file once and builds a modulePath → SourceAnalysis map. When generating tests for payment.go, it can pull the public API signature of discount.go (which payment.go imports) from memory:

// In the prompt:
// Internal dependency signatures:
// discount.ApplyPromoCode(order Order, code string) (Order, error)
// discount.ValidateCode(code string) bool

Enter fullscreen mode Exit fullscreen mode

This tells the model what the real interface looks like so it generates test doubles that match the actual signatures — not invented ones.

In watch mode, when a file changes, only that file's entry is refreshed. The rest of the index stays warm between regens.

Why the Separation Matters

The two layers solve different problems:

  • Agent context is about development-time navigation. It's hierarchical, human-readable, and loaded selectively. It describes architecture and invariants. It lives in the repo and is maintained alongside the code it describes.

  • Runtime LLM context is about generation-time quality. It's merged from two levels, injected into system prompts, and exempt from token budgets. It describes conventions and patterns specific to the target project — things an LLM can't infer from source code alone.

Conflating the two leads to either bloated system prompts (dumping agent context into every generation request) or under-informed agents (giving them only the user-facing conventions doc with no architectural guidance). Keeping them separate means each audience gets exactly what it needs.

Next: the cross-platform bugs we hit shipping a Go CLI — detector boundary escapes and Windows path separators.