Your AI agent is paying to send the same file dump five times. tokdiet is a local proxy that sits between your agent and the model API, meters every token, puts your bloated context on a diet — and proves the answer didn't get worse.

ccusage that shrinks the bill — without losing quality.

The proof (this is the whole point)

Every "context optimizer" cuts tokens. The scary question is the one they can't answer:

"If I cut the context, does the model get dumber?"

So we measured it. A 66-task A/B benchmark across 6 categories on a real model (MiniMax‑M3), each task run twice — full context (baseline) vs through tokdiet (governed) — graded against the known answer, repeated ×3 and majority‑voted to cancel model noise:

                       baseline      tokdiet
  input tokens          5.07M    →    1.46M       −71%
  quality (66 tasks)     64/66        63/66        ≈ parity (95–97%)
  ─────────────────────────────────────────────────────────
  198 paired runs · LLM-judge 92% similarity · confirmed on a 2nd model (MiniMax-M2.5: −72%)

−71% tokens, quality on par with baseline. Real requests, real grading — not a mock. The ~1–2 task gap is model nondeterminism plus the model declining to echo a secret — not context loss; the hardest "needle buried in junk" adversarial cases pass, because tokdiet doesn't delete blindly — it pages cold context out recoverably and protects anything on‑topic. Reproduce it yourself: node bench/run.mjs (needs an API key in env).

How it compares

Everyone shows the bill or cuts it blind. tokdiet is the one that cuts it and proves the model didn't get dumber — and stops cutting the moment it might.

Quick start

# 1. Start the proxy (and live dashboard) — no install needed
npx tokdiet start

# 2. Point your agent at the proxy instead of the real API
export ANTHROPIC_BASE_URL=http://localhost:7787
export OPENAI_BASE_URL=http://localhost:7787/v1

Now run your agent (Claude Code, Cursor, Codex, your own script) as usual. Traffic flows through tokdiet, gets metered and compacted, and is forwarded upstream unchanged in every way that matters.

Your API key stays with you. tokdiet reads x-api-key / Authorization only to forward them upstream. They are never written to SQLite and never written to any log. And it's fail‑open: if anything inside the governor errors, it falls back to transparent passthrough — the proxy will never break your request or surface its own 5xx.

Default ports: proxy 7787, dashboard 7878. Override with --port / --dashboard-port.

Install via Claude Code

tokdiet ships as a Claude Code plugin via its own marketplace:

/plugin marketplace add agiwhitelist/tokdiet
/plugin install tokdiet

What the plugin does — and what it doesn't. The plugin ships a lightweight metering hook plus a /tokdiet command. The hook runs on every tool call (PreToolUse + PostToolUse) and logs tool I/O byte sizes to ~/.tokdiet/tool-meter.log. It does not save tokens by itself — a plugin can't set ANTHROPIC_BASE_URL for the Claude Code process, so it can't route your traffic through the compacting proxy.

The actual token savings come from the proxy. Start it and point Claude Code at it (this is what gives you the ~−71% token reduction):

npx tokdiet start
export ANTHROPIC_BASE_URL=http://localhost:7787   # then launch Claude Code from this shell

View metered tokens, cost, and savings any time with npx tokdiet report, or run /tokdiet inside Claude Code for these instructions.

Works with Claude Code (and it's careful about it)

Claude Code is the flagship use case, and it has two landmines a naive compacting proxy walks straight into. tokdiet handles both:

• Prompt caching. Claude Code marks a cached prefix with cache_control; cached input costs ~10% of normal. Rewriting that prefix invalidates the cache and can make a request cost more. tokdiet is cache‑aware — it never touches content at or before a cache_control breakpoint.
• Extended thinking. Claude Code sends signed thinking blocks that Anthropic requires returned verbatim; touching one is an instant 400. tokdiet is thinking‑safe — signed/thinking blocks are never surfaced or mutated.

Both are covered by regression tests (tests/cc-compat.test.ts).

A note on honesty: the dollar‑savings story applies to pay‑per‑token API keys (MiniMax, Anthropic API, OpenAI, …). On a flat Claude subscription there are no per‑token charges to cut, so the value there is metering, budgets, and the live dashboard — not dollars.

How it works

tokdiet is a streaming reverse proxy. SSE responses are proxied incrementally (never buffered whole), so your agent's tokens still stream in real time.

                            tokdiet (localhost:7787)
   agent  ─────────────────────────────────────────────────────────────►  model API
 (Claude  request    ┌───────────┐  ┌───────┐  ┌────────┐  ┌───────────┐   (Anthropic /
  Code,  ──────────► │interceptor│─►│ meter │─►│ budget │─►│ compactor │──►   OpenAI /
  Cursor, raw key    └───────────┘  └───────┘  └────────┘  └─────┬─────┘      Gemini /
  Codex,  forwarded   detect          count      session/        │ dedup / elision /  MiniMax)
  …)                  provider,       tokens     day / repo      │ mid-summarize
                      keep body        & cost     limits          ▼
                      byte-faithful                          ┌───────────────┐
   response                                                  │ quality guard │
 ◄──────────────────────────────────────────────────────────┤ shadow-eval + │
   streamed back, token-for-token                            │  safe-mode    │
                                          ┌──────────────┐   └───────┬───────┘
                                          │ store(SQLite)│◄──────────┘
                                          │ + dashboard  │  telemetry, savings, degradation
                                          └──────────────┘

Context as virtual memory (the idea)

Blind compaction is "delete and pray." tokdiet treats your context like virtual memory: hot content (recent, pinned, relevant to the current question) stays resident; cold content (stale, redundant) is paged out to a local store as a recoverable stub — not deleted. The full block is kept in SQLite keyed by an id, so it can be audited and (roadmap) paged back in on demand when the model actually needs it.

The 3 quality mechanisms

Compaction strategies (safest‑first)

1. Dedup — loss‑free. When the same large block is re‑pasted across a conversation, keep the freshest copy verbatim and replace earlier copies with a pointer marker. Works on near‑duplicates too (a file re‑pasted with a few lines changed), not just byte‑identical ones.
2. Elision — recoverable. Page out the bulk of old tool results (file dumps, command output), keeping a preview plus the salient lines (errors, ids, KEY=VALUE, URLs, paths, numbers) and storing the full body for recovery. Recent, pinned, and question‑relevant results are kept intact.
3. Mid‑summarize (off by default) — summarize mid‑history with a cheap model. Opt‑in (it costs money).

Commands

tokdiet <command> [flags]   # alias: td

Configuration

Run tokdiet init to create tokdiet.config.json, or pass one with --config. All fields are optional and merge over sensible defaults.

Upstream overrides (point at a non‑default origin — e.g. MiniMax): TOKDIET_ANTHROPIC_UPSTREAM, TOKDIET_OPENAI_UPSTREAM, TOKDIET_GEMINI_UPSTREAM (legacy CTXGOV_*_UPSTREAM still read for back‑compat).

Dashboard

With the proxy running, open http://localhost:7878 — a single self‑contained page that streams live updates over SSE (loopback only; your cost data never leaves the machine):

┌─ tokdiet ─────────────────────────────────────────  ● live · :7878 ─┐
│                                                                       │
│  SESSION  claude-code › my-repo › MiniMax-M3                          │
│  context  ███████████████████░░░░░░░░░░  64%   128,402 / 200,000 tok  │
│                                                                       │
│  ┌── TODAY ────────────────┐   ┌── SAVED (cumulative) ─────────────┐  │
│  │ sent     1.43M tok       │   │  $12.40  ▁▂▃▅▆▇█  ↑ saving $1.07/h │  │
│  │ saved    3.64M tok       │   │  3.6M tokens never left this box  │  │
│  │ spend    $0.43           │   │  −71.8%  on real traffic          │  │
│  └──────────────────────────┘   └───────────────────────────────────┘  │
│                                                                       │
│  QUALITY GUARD   measured degradation 0.4%  ┃▏▏▏▏▏▏▏▏░░┃ budget 2.0%  │
│                  ▁▁▂▁▁▁▂▁▁▁  72 shadow-evals   safe-mode ● ON · OK     │
│                                                                       │
│  STRATEGY LEADERBOARD            fires    tokens saved     Δ quality   │
│   ▸ dedup          ███████████    312       1.91M           +0.0%      │
│   ▸ elision        ██████         168       1.42M           +0.6%      │
│   ▸ midSummarize   · off ·          0          —              —        │
│                                                                       │
│  BY TOOL   claude-code ██████████ $0.31   cursor ███ $0.09  codex ▍$03 │
└───────────────────────────────────────────────────────────────────────┘

Five live screens: Live session, Savings, Quality (degradation + safe‑mode status), By tool & repo, and Strategy leaderboard — all updating in real time over SSE.

See the savings — no API key required

npm run build && node scripts/demo.mjs

Stands up a mock Anthropic upstream on loopback, starts the real tokdiet proxy in front of it, and sends one realistic bloated agent request through the whole pipeline — actual interceptor, tokenizer, compactor, pricing, telemetry, and shadow‑eval. No external network, no real key. It prints a before/after table proving the input shrank while the answer stayed identical (so shadow‑eval reports ~0% degradation). (The scenario is synthetic; your real savings depend on how much your own conversations repeat.)

Supported providers

Prices come from pricing.json (USD per 1,000,000 tokens, dated, user‑updatable, hot‑reloaded on start; exact match then longest‑prefix).

Roadmap

• Page‑fault auto‑reinjection — when the model references a paged‑out id or signals it's missing content, restore it and retry automatically (partially shipped).
• Semantic dedup (shipped) — near‑duplicate collapsing.
• Embedding judge — local semantic scoring instead of the heuristic.
• Self‑calibrating policy — learn safe aggressiveness per repo from shadow‑eval outcomes.
• Quality ledger — auditable before/after + measured‑degradation record.

See docs/DESIGN-context-virtual-memory.md for the full design.

Limitations & honesty

• The default judge is a heuristic (word/char similarity), not a semantic oracle. Switch shadowEval.judge to "llm" for a model‑graded score. Embedding judge isn't implemented yet.
• Shadow‑eval costs money — it's a real extra upstream request, so it's sampled (5% default) and its cost is reported separately.
• Session inference is heuristic — per‑session/per‑repo attribution is inferred from request metadata.
• Page‑fault recovery is limited for streaming responses.
• Cost figures are estimates — only as accurate as your pricing.json.

License

MIT