by Alem Tuzlak on Mar 12, 2026.

If you've ever built a production AI application, you know the pain. Your chat() call starts simple — then you need logging, then content filtering, then tool caching, then rate limiting, then an audit trail... and suddenly your clean server endpoint is a 200-line monster with deeply nested try/catch blocks.

TanStack AI now ships a first-class middleware system for the chat() function. It's composable, type-safe, and comes with batteries included via the new @tanstack/ai/middlewares subpath export.

Let's dive in.

A middleware is just an object with optional hooks. No classes, no inheritance, no decorators — just a plain object that satisfies the ChatMiddleware interface:

Drop it into the middleware array and you're done:

No wrapping, no monkey-patching, no provider-specific hacks.

The middleware system exposes hooks at every meaningful point in the chat() lifecycle:

Every hook receives a rich ChatMiddlewareContext with requestId, iteration, chunkIndex, current messages, and two powerful control functions: abort() to stop the run, and defer() to register non-blocking side effects that run after streaming completes.

Before we look at code, let's talk about what this actually unlocks.

Transform Anything the Model Sees

onConfig fires at two critical moments — once at startup (phase: 'init') and once before every model call (phase: 'beforeModel'). Return a partial config and it's shallow-merged in. You can change messages, systemPrompts, tools, temperature, maxTokens, metadata, or modelOptions — per request, per iteration, conditionally.

This means you can inject tenant-specific system prompts based on who's making the request. You can strip tools out after the first iteration so the model stops looping. You can bump temperature on retries. All without touching your adapter or your tool definitions.

When multiple middleware define onConfig, the config is piped through them in order — each one sees the merged result of the previous. So you can layer concerns: one middleware handles auth-based prompt injection, another handles tool filtering, a third adjusts model parameters. They compose naturally.

Intercept Every Streamed Chunk

onChunk gives you access to every piece of data the adapter yields. You can observe it, transform it, expand it into multiple chunks, or drop it entirely by returning null.

This is where content filtering lives. Redact PII before it reaches the client. Strip out markdown formatting your UI doesn't support. Inject synthetic chunks to add custom metadata to the stream. If a previous middleware drops a chunk, downstream middleware never see it — the pipeline is clean.

Control Tool Execution Without Touching Tools

onBeforeToolCall and onAfterToolCall wrap every tool invocation. Before a tool runs, you can:

• Pass through — return void, the tool runs normally
• Transform arguments — return { type: 'transformArgs', args } to rewrite what the tool receives
• Skip execution — return { type: 'skip', result } to short-circuit with a synthetic result (this is how caching works)
• Abort the entire run — return { type: 'abort', reason } to stop everything

This is first-win composition: the first middleware that returns a non-void decision wins, and the rest are skipped for that call. After execution, onAfterToolCall fires on all middleware with timing data, success/failure status, and the result or error.

onToolPhaseComplete fires after all tool calls in an iteration are done, giving you aggregate data — what completed, what needs user approval, what needs client-side execution. This is where you'd implement batch-level validation or summary logging.

Track Token Spend in Real Time

onUsage fires once per model iteration with promptTokens, completionTokens, and totalTokens. Combined with ctx.iteration, you can build cumulative budgets, per-model cost tracking, or real-time spend dashboards without any external instrumentation.

Abort from Anywhere

ctx.abort(reason) is available in every hook. Call it and the run stops gracefully — onAbort fires as the terminal hook with the reason and duration. This is how you implement timeouts, budget limits, content policy violations, or any "stop everything" signal.

Fire-and-Forget Side Effects

ctx.defer(promise) registers a non-blocking promise that executes after the terminal hook. It never blocks streaming. This is the right pattern for analytics, audit logs, webhook notifications, or database writes that shouldn't add latency to the user's response.

Thread Request-Scoped Data

The context option on chat() is an opaque value that flows into ctx.context on every hook. Pass in your auth token, tenant ID, feature flags, or any request-scoped data — middleware reads it without coupling to your HTTP framework.

Middleware compose cleanly in array order with well-defined semantics:

This means you can stack a content guard before a logger and the logger only sees the redacted output. Or put a rate limiter before a cache and the cache never stores rate-limited calls. The ordering is explicit, the rules are simple, and there are no surprises.

The new subpath export ships two production-ready middlewares. You don't need to write these yourself — they're tree-shakeable, so unused ones don't end up in your bundle.

toolCacheMiddleware — Never Re-Execute the Same Tool Call

If your agent calls getWeather({ city: "Berlin" }) three times in a conversation, do you really want to hit the API three times?

Under the hood, it uses onBeforeToolCall to check the cache and returns { type: 'skip', result } on a hit — the tool never executes. On a miss, onAfterToolCall stores the result. Failed calls are never cached.

Custom cache keys let you ignore irrelevant arguments:

Pluggable storage means you can swap the in-memory LRU map for Redis, a database, or anything with getItem/setItem/deleteItem:

contentGuardMiddleware — Real-Time Stream Redaction

Your model just streamed back a user's SSN. Your model just leaked an internal API key. Your model included PII in a customer-facing response.

Two strategies are available:

• delta — applies rules to each chunk as it arrives. Zero added latency, but patterns split across chunk boundaries might slip through.
• buffered (default) — accumulates content and applies rules to settled portions, holding back a configurable look-behind buffer. Catches cross-boundary patterns at the cost of a tiny delay.

Set blockOnMatch: true to drop entire chunks when any rule matches instead of replacing:

The middleware system is deliberately low-level and composable. Here are patterns that fall right out of the hook design.

Per-Iteration Tool Swapping

Expose different tool sets at different stages of the agent loop — let the model search first, then unlock write tools:

Token Budget Enforcement

Track cumulative usage and abort when you've spent enough:

Deferred Analytics (Non-Blocking)

Fire-and-forget side effects that never slow down streaming:

Dangerous Tool Guard

Intercept tool calls and block the ones you don't trust:

Tenant-Aware Prompt Injection

Thread request-scoped data through middleware via context:

Stacking It All Together

The real power is composition. Here's what a production setup might look like:

Six concerns. Six middleware. Zero coupling between them. The order is explicit and the composition rules are predictable.

Middleware events are wired into TanStack DevTools out of the box. Every hook execution, config transformation, and chunk transformation emits structured events via @tanstack/ai-event-client. The new Iteration Timeline and Iteration Card UI components let you visually trace what happened at each step of the agent loop — which middleware fired, how long each hook took, and what was transformed.

This isn't just logging. It's a full timeline view of your middleware pipeline, per iteration, in real time.

All middleware types are exported from the main package:

Built-in middleware lives in the tree-shakeable subpath:

The full middleware documentation is available in the TanStack AI Middleware Guide.