In April 2026, four months into the year, Uber ran out of its entire annual AI budget. Not because of an outage or a security incident, but because its engineers were using Claude Code exactly the way the company wanted them to. Adoption climbed from roughly a third of the organization to nearly all of it within weeks, and the bill followed the same curve. By the time CTO, Mr. Praveen Neppalli Naga, went public about it, the company was, in his own words, back to the drawing board.
Nobody made an obvious mistake here. The tool worked, and engineers shipped more code, faster. What broke was the assumption underneath the budget that AI spend would carry a fixed, predictable, per-seat cost. Only it doesn’t. It behaves more like a utility bill that scales with how hard you lean on it, because large language models charge for every token.
That is the story quietly repeating itself across enterprise AI right now. However, reducing token usage isn’t a one-time cleanup pass. It is a discipline that outlines how you write prompts, how much context each call carries, what you cache and how tightly you control what comes back. Teams that treat it this way tend to compound their savings over time. Teams that treat it as a pricing problem, hoping a cheaper model would quietly fix things, tend to get blindsided the way Uber did.
In this article, we will walk you through eight practical strategies enterprise teams are using to bring token consumption under control. Now let’s get into it without any further ado.
Tokens are like small chunks of text. They are often pieces of words rather than whole words that a model reads, processes, and gets billed for.
As a rough rule of thumb, one token works out to about four characters of English text. Thus, a phrase like “reducing token usage” comes out to roughly seven tokens, not three words.
Uber’s experience earlier this year is the clearest illustration of why this matters. The company burned through its full annual AI budget in four months.
However, by pairing prompt caching, smarter model routing and usage dashboards, Uber quadrupled its AI usage while bringing per-token costs back down. This clearly demonstrates that the problem wasn’t the tool. It was the absence of the kind of cost architecture this article covers.
AI cost management has gone from a niche concern to something 98% of FinOps teams now handle.
And when the FinOps Foundation asked practitioners what capability they most need but still don’t have, the top answer was granular visibility into AI spend, broken down by token, request, and feature. Most organizations aren’t struggling to see that they are spending on AI. What they are missing is the ability to see where the spend is going closely enough to actually act on it.
Token consumption usually comes from a handful of structural habits that compound quietly across every request. This includes:
Before implementing any strategies to reduce token usage, a team needs a clear picture of where its tokens are actually going.
So start with the honest baseline: a large share of AI teams don’t track LLM costs beyond that monthly invoice, which means the first real fix isn’t a strategy at all, it is basic instrumentation. From there, usage should be broken into input, output, cache-read, and cache-write tokens rather than one blended number.
The next step is attribution. This involves tying that usage to a specific feature, user, or request type. This answers which part of our product is truly expensive.
None of this needs to be sophisticated to be useful. Even a simple weekly or monthly review, treated as seriously as a cloud infrastructure bill, is usually enough to catch a cost trend while it is still small.
This is the cheapest lever, and usually the first one worth trying because it costs
nothing beyond a rewrite. It just involves a clean prompt that says what it needs to say and nothing more.
Here’s how it works!
Write clear and concise prompt: Vague prompts don’t just risk a wrong answer. They often trigger a follow-up clarification or a second pass to fix the output, and both cost as many tokens as the original request did. A prompt that states the task, the constraints, and the expected format in plain language up front tends to get it right the first time.
Eliminate redundant instructions: Production prompts that have grown over months of iteration often say the same thing twice in different words. They simply restate context the model already has. Every duplicated instruction gets billed twice. Auditing a live prompt for repetition is often the single highest-leverage thing a team can spend on cost.
Structure prompts efficiently: Your prompt’s format choice matters more than you expect. JSON is the default for passing structured data into a prompt. However, its brackets, quotes, and repeated keys add real overhead, especially tabular data like product catalogs or log excerpts. TOON (Token-Oriented Object Notation), a serialization format that emerged in late 2025 specifically for LLM input, cuts token counts by roughly 30–60% against JSON, while matching JSON’s retrieval. It is still an evolving format, but a genuinely useful option for any prompt carrying rows of structured data.
Use dynamic prompt templates: A single static prompt built to cover every possible scenario tends to accumulate instructions for edge cases that apply to only a small fraction of requests. Templating that logic and using it only when the specific request actually needs it keeps prompts optimized.
This strategy controls all the attachments that your prompt carries along with the instructions, such as retrieved documents, conversation history and tool outputs. All of that gets bundled in and billed as tokens too, even though you didn’t type any of it yourself.
Here’s how to optimize this context window size:
Retrieve only relevant context: Most RAG systems tend to retrieve context generously, pulling in extra chunks in case the model needs them. This habit gets expensive too fast. And it usually doesn’t even improve the answer quality. To minimize this, Anthropic suggests a slightly counterintuitive step. It says if a knowledge base is small enough, under roughly 200,000 tokens or about 500 pages, skip the retrieval layer entirely and include the whole thing in the prompt, paired with caching to keep it affordable.
Summarize conversation history: AI models don’t remember past messages on their own. Every time you send a new message in a chat, the entire conversation so far has to be resent along with it because that is the only way the model knows what was said before. The solution to this is, instead of carrying the full word-for-word history forward, periodically compress older messages into a short summary and only keep the last few messages in full. That keeps the cost from snowballing.
Chunk documents effectively: When a big document needs to be searchable, it gets cut into smaller pieces called chunks. This helps the system to pull just the relevant piece instead of the whole document. However, splitting a document into chunks is tricky. Chunks that are too large carry token overhead the query doesn’t need, whereas chunks that are too small lose the surrounding context that made them meaningful in the first place. Getting this right is never easy and takes practice.
Remove duplicate information: Sometimes a search step pulls back the same information twice in slightly different forms. A quick check before sending everything to the AI catches these repeats. Nobody notices duplicate content is being sent unless they specifically go look, so it tends to sit there wasting tokens indefinitely.
Often, two different tricks get lumped under this one name: Prompt caching and semantic caching. The former, built into Anthropic, OpenAI, and Bedrock, lets the AI provider remember the repeated opening part of a request, like your standing instructions, so it doesn’t reprocess that part every time. Semantic caching works differently. It checks whether a new question means roughly the same thing as one already answered, even if it is worded differently. If yes, it reuses the old answer instead of calling the AI again. Most systems in production use both for different parts of the job.
Here’s how to implement this:
Cache frequently asked queries: This is exactly what semantic caching is built for. Support bots and FAQ assistants field the same handful of underlying questions, phrased a hundred different ways. Now instead of treating each phrasing as a brand-new question, the system recognizes the repeat and reuses the existing answer.
Cache retrieval results: In systems that search a document library before answering, the same source material often gets pulled for several related questions. Caching that retrieved material, not just the final answer, means the search step only has to run once per topic, not once per question.
Cache agent responses: AI agents often work through a task in many small steps and resend the same setup information at every single step. That repetition is exactly what prompt caching is built to catch. ProjectDiscovery, which runs an AI security-testing tool through 20–40+ steps per task, saw this firsthand. After fixing the repetition, their cache hit rate jumped from 7% to 84%, cutting total AI costs by 59–70%.
Not every request needs your most powerful and most expensive model. Routing sends each request to the cheapest model that can still handle it well, saving the expensive one for questions that actually need it.
Here’s how this is achieved:
Route simple tasks to smaller models: Simple, well-defined requests such as short lookups and basic Q&A often get answered just as well by a smaller, cheaper model. Since smaller models cost a fraction per token, this is where most of the savings come from.
Escalate complex tasks to advanced LLMs: Reasoning-heavy or high-stakes requests still need the strongest model available. A good router flags these and sends them up, rather than risking a weak answer to save a few tokens.
Hybrid model strategies: Most production systems use both. A lightweight router decides in real time which model handles each request. AISquared’s Bolt model family works this way inside the UNIFI platform. Bolt Instruct models sit in front of every request and decide in real time whether it needs a small, fast model or should it escalate to something larger.
RAG systems are only as efficient as what they choose to retrieve. Most of the waste here comes from pulling in more and lower-quality context than the question actually needs.
The key here is to:
Retrieve less, retrieve better: More retrieved documents doesn’t mean a better answer. Often, it just means more text for the model to sift through and at a much higher cost. More targeted results, on the other hand, usually work just as well. Additionally, how a query gets matched against the document library matters more than how many results come back. AISquared’s Bolt Embedding helps to retrieve better context for enterprise questions, which improves the quality of downstream answers.
Filter low-value documents: Not everything retrieved is worth including. A final relevance check before assembly and dropping anything below a quality threshold helps to keep only what is actually likely to help answer the question.
Output tokens usually cost more per token than input tokens. They are also the easiest to lose control of. When left unconstrained, a model tends to explain itself far more than the answer actually needs.
Here’s how to control this:
Set maximum output tokens: A hard cap on response length using the “max_tokens” setting most providers offer is the simplest way to avoid a response that runs long for no reason. The trick is calibrating it to the task rather than picking a safe, generous round number for all responses. For instance, a classification answer might just need 10 tokens while a summary might need 500. Using the same ceiling for both either wastes budget on short answers or risks cutting the long ones off mid-sentence.
Request structured responses: Asking for a fixed format, such as a set of fields, a table, a short list, etc., tends to produce shorter answers than open-ended prose. It is because there is no room left for filler explanation around the actual content. The only caveat here is that sometimes an overly detailed schema, packed with long field descriptions, can add back more tokens in the request than the structure saves in the process.
Avoid overly verbose outputs: Small instructions go a long way here. Tell the model to skip restating the question, skip the preamble, and skip hedging disclaimers unless they are really needed. Verbosity is usually a default habit and not something the task requires.
Multi-agent systems, where several AI agents work on pieces of a task and report back, are powerful. But that power comes at a real cost. Anthropic’s engineering team found that a single agent uses roughly 4x the tokens of a normal chat turn, and a multi-agent system uses roughly 15x. The goal here isn’t to avoid multi-agent setups, but to keep that overhead as low as possible.
Here’s how to do that:
Minimize agent-to-agent communication: Every back-and-forth between agents costs tokens on both ends. Giving each agent a complete, self-contained task description and clear success criteria up front avoids the clarifying question loops that quietly rack up cost.
Share context efficiently: The default way agents share information is by passing full findings back and forth through a central coordinator. This means the same content gets repeated across every handoff. A more efficient pattern is when agents write their full findings to shared storage and pass back only a short reference to where that information lives. The next agent pulls the detail only if it actually needs it.
Eliminate duplicate processing: When multiple agents work on a task independently, it is common for two of them to end up doing the same research without realizing it. Assigning clear ownership, such as one agent responsible for retrieval, others reusing its output instead of re-querying on their own, etc., prevents that overlap from happening in the first place.
Everything mentioned above only pays off if a team can actually see where its tokens are going. Without that visibility, even a well-optimized system can drift back into waste over a few months, simply because nobody is watching.
Here’s how to implement this:
Token analytics: Simply tracking the total tokens number isn’t enough. You must break usage down by input, output, cache-read, and cache-write tokens, and further, by feature, endpoint, or model. This provides an exact idea about which part of the system is actually expensive.
Cost dashboards: Tools like Helicone, Portkey, and Langfuse sit between an application and the AI provider, turning raw token counts into cost-per-feature and cost-per-user views in something closer to real time. This provides visibility that updates daily instead of showing up as a surprise on next month’s invoice.
Usage alerts and budgets: A dashboard someone has to remember to check is only half the fix. Pairing it with alerts and hard budget caps per team or feature turns monitoring from a passive report into something that actually catches problems while they are still small.
Audit prompts for redundancy: Cut duplicated instructions and restated context from the prompt. A five-minute review often finds the easiest savings in the whole system.
Match format to data type: Use compact, structured formats like TOON for tabular or repetitive data instead of defaulting to JSON everywhere.
Retrieve only what is needed: Skip RAG entirely for small knowledge bases. For larger ones, pull fewer, more targeted chunks rather than a generous batch.
Summarize instead of resending: Compress older conversation turns into a rolling summary rather than carrying the full transcript forward on every call.
Route by task complexity: Send simple requests to smaller models and reserve the frontier model for what actually needs it.
Cap and structure outputs: Set “max_tokens” deliberately per task and request structured formats instead of open-ended prose where possible.
Design multi-agent systems around shared references: Avoid repeated context passed through every handoff.
Track usage by category and set alerts: Break down input, output, and cache tokens by feature, and pair dashboards with budget caps so overruns get caught early.
Here are the most common mistakes enterprises make that shoots up their token usage:
Treating cost as a pricing problem, not an architecture one: Switching to a cheaper model without fixing bloated prompts or unbounded context usually just delays the same overrun.
Retrieving more than needed: Pulling extra documents or context that might be relevant adds tokens without reliably improving answer quality.
Resending full conversation history indefinitely: Without summarization, a long-running chat gets more expensive with every single turn.
Skipping similarity threshold tuning: A cache set too loose starts returning confidently wrong answers to genuinely different questions.
Defaulting to the most powerful model for high accuracy: Routing everything to the top-tier model regardless of task complexity is one of the biggest and most avoidable sources of overspend.
Leaving output length unrestrained: Models default to explaining themselves more than the task requires unless explicitly told not to.
Not tracking usage until the invoice arrives: Without ongoing visibility, cost problems get discovered a month late, when there is nothing left to do.
Several of the strategies above show up as built-in features inside AISquared’s UNIFI platform.
AISquared’s Bolt model family is a suite of open-weight, purpose-built models. It is designed specifically to address the enterprise “token problem,” i.e., using large frontier models for routine work that doesn’t need frontier-level reasoning.
Inside UNIFI, Bolt Instruct models serve as both a guardrails layer and a routing layer, automatically directing each request to the smallest model capable of handling it well and reserving larger models for genuinely complex tasks.
The numbers behind this are concrete. In a report, AISquared tested a Bolt-powered routing layer against a monolithic single-model deployment. The routed setup shifted 58% of requests to smaller, lower-cost models, cutting total operating cost by 49% and improving average latency by 42%.
On document processing specifically, Bolt’s specialized models handled invoice parsing at roughly one-twentieth the cost of frontier alternatives while improving accuracy. AISquared estimates this saves an enterprise processing a million invoices a month close to $1.9 million annually.
Beyond routing, UNIFI’s broader architecture touches several of the other strategies covered here. Bolt Embedding models support more targeted retrieval for RAG, and UNIFI’s built-in governance and feedback layer gives teams the kind of usage visibility every AI system needs.
Token usage isn’t a line item to trim after launch. It is, in fact, an architectural choice that shapes whether an AI feature scales sustainably or turns into a budget surprise. All strategies mentioned here work because they compound together. Each one makes the next more effective, turning token spend from an unpredictable variable into something a team can actually plan around. Platforms like AISquared increasingly help enterprises to put these strategies into practice.
Prompt caching stores the repeated part of a request so a provider doesn’t reprocess it every time. It only works on identical text. Semantic caching matches queries by meaning, so differently worded questions with the same intent can still hit a cached answer.
It depends on the use case, but production examples are substantial. Prompt caching alone can cut costs on repeated content by up to 90%, and intelligent model routing has reduced total costs by 30–49% in real deployments, without a meaningful drop in output quality.
Not when it is done well. Routing is designed to send only tasks a smaller model can genuinely handle while reserving frontier models for complex reasoning.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。