Bigger Context Windows Solved One Problem And Created Another

Tokens are more than just a language construct that is loosely aligned with the information you feed into and out of a model. Tokens are the unit by which model memory, time and cost get rationed. Anthropic’s documentation defines tokens as the smallest unit Claude processes and notes that one token is roughly 3.5 English characters on average. Every prompt, every file, every tool trace, every retry and every long reply adds to the context and pushes you closer to the hourly or weekly limits on your flat-rate subscription plan, or adds to the metered bill if you’re interacting through an API. Usually those limits are not much of an issue for simple single prompts or basic interactions, but they can be a real problem when a coding agent drags an entire codebase and swollen history through every turn.

Context windows are the measure of how much information a model can understand as part of a conversational interaction. The rush toward giant context windows made sense as developers wanted models that could read more code, hold more instructions and keep more state alive during multi-step tasks. Anthropic now offers a 1 million token context window in beta for some newer Claude models on requests that exceed 200,000 input tokens.

For years, model vendors treated ever-larger context windows as a selling point. The reality is more complicated. A larger window can tempt users to stop trimming instructions and files. Some teams start tossing in whole repos, long meeting notes and large instruction files, trusting the model to sort it out. Tool outputs then pile up turn after turn. Old plans, stale constraints and repeated explanations remain in memory and clutter the next request. Model contexts become jammed with information like a junk drawer.

Long context is no longer just a bragging right, but a billable feature that can offer more flexibility at a higher cost. From a practical perspective, Anthropic’s own Claude Code material shows how token costs impact developers who use AI on a daily basis. The company says average usage costs around $6 per developer per day, with most developers staying under $12 and monthly team costs often falling in the $100 to $200 range per developer, depending on how the tool is used. The same pattern is showing up across the market.

OpenAI’s posted prices list GPT-5.4 at $2.50 per million input tokens and $15 per million output tokens, with cached input priced much lower. Anthropic lists Claude Sonnet 4.6 at $3 per million input tokens and $15 per million output tokens, with batch processing priced at half those rates. Those numbers suggest that repeat or deferred inference can cost vendors far less than premium real-time use, even as long-context and agentic workloads remain expensive to support.

At the same time, the vendors’ own product changes show that serving these workloads carries real costs. What customers pay is likely higher than the vendors’ raw per-request serving cost, though that spread has to cover far more than GPU time alone. Anthropic now pushes heavy Claude users into separately billed “extra usage,” and OpenAI breaks out separate charges for tools, containers and long-running Codex work, which would make little sense if frontier inference were already cheap to provide at scale. Epoch AI estimates that capability-adjusted inference costs have been falling by roughly 5 to 10 times per year. So while vendors appear to price above raw inference cost, they are doing so in a market where the underlying cost curve is dropping fast and where long-context agents still put real pressure on margins and capacity.

Are The Vendors Changing Limits Or Are We Just Hitting Them Faster?

Some people think there’s a conspiracy afoot to reduce subscription limits or somehow play with the rate at which tokens are being consumed. In some cases, yes, the vendors are adjusting limits. OpenAI says ChatGPT Plus usage caps may vary during periods of high demand and based on system conditions, so part of the frustration is real that the ceiling itself is not always fixed. So users are not imagining the squeeze. In some systems, the formal cap can vary.

The bigger reason for hitting limits faster than ever, though, is that people are asking the models to do much more work per session than they were a year ago. OpenAI says Codex burns through allowances faster on larger codebases, long-running tasks and extended sessions that require the model to hold more context. OpenAI’s Codex cost and subscription usage depends on model choice, task size, whether work runs locally or in the cloud and how much context the system needs to carry. The company also says directly that larger codebases, longer-running work and longer sessions with more context consume more per message.

Anthropic has made a similar point from another angle, saying Claude usage varies with message length, file attachment size, current conversation length and tool usage. In others, the same session simply consumes the allowance faster when it includes longer context, larger files or heavier tool use.

In other words, most users are not suddenly getting less efficient models or stingier vendors looking to push extra billing. They are using the models in more agent-like ways, with longer memory, more files, more tools and more back-and-forth packed into each task. The result feels like shrinking limits, even when the real change is that every session has become heavier, costlier and harder to serve.

Furthermore, Anthropic has taken a harder stance against token-heavy third-party systems such as OpenClaw. Instead of lowering caps for heavy users, the company stopped letting standard Claude subscription limits cover third-party agent harnesses such as OpenClaw, pushing those users toward separately billed API or extra-usage paths after saying those tools created an “outsized strain” on its systems. That suggests Anthropic is drawing a sharper distinction between its own Claude product usage and outside agent frameworks that keep models running for long stretches.

That is the backdrop for the recent public frustration around AI usage. Users are not only hitting message caps, but they are also discovering that AI systems become expensive the moment they stop behaving like simple chatbots and more like software workers. With product releases such as Claude Cowork and agentic tools that hit token usage hard, this is becoming a pressing issue.

Why Use More Token When Less Do Trick?

Anthropic’s own support guidance now reflects the reality of heavy token usage. It warns that tools and connectors can be token-intensive and tells users to pare back project instructions, trim files and limit what stays in the conversation. In other words, the company is quietly teaching users to do what experienced engineers have always done with scarce resources: curate inputs. The real significance of Anthropic’s new usage is not just that heavy users may pay more, but also that the company has started admitting, in product terms, that not all usage is created equal.

A single chat question is cheap, but a persistent coding agent that reads a codebase, edits files, runs commands, loops on failures and narrates each step is not. Agentic coding tools can work across files and tools, understand a codebase and automate development tasks. This might be extremely useful but not particularly subscription-friendly that has hard usage caps that reset every few hours or weekly.

Users are looking for creative ways around these limits. In a recent viral Reddit post, a user claimed that talking like a caveman reduced token usage and helped stretch subscription limits. The trick was to instruct the model to answer in very short, stripped-down sentences with no filler, pleasantries or narration in hopes of cutting output-token use, as one Redditor said: "Why waste time say lot word when few word do trick?”. The meme spread because it captured the real frustration that users are now so aware of token burn that they are changing the model’s speaking style to make sessions last longer.

Increasing Use of Local Models

Another approach to solving the subscription token issue is to use models you run and host yourself. Local models provided by a range of open source and third party vendors that can run on systems such as Ollama appeal to anyone tired of paying a vendor every time a model reads the same files again. Run the model on your own hardware and the token meter, at least in the vendor sense, fades into the background.

The tradeoff of course is that you are limited by your system’s memory and processing power. Context length depends on available VRAM. As you add agents, web search and coding tools the need for larger context grows and system resources become stretched. That shifts the pain point. With Claude or Codex, the question might be whether you are about to exceed your included usage or should pay for extra usage. With local models, the question is whether your machine has enough capability to keep the model useful at the context length your workflow needs.

Local models make the most sense when the work is repetitive, private, needs access to local data or is steady enough that buying hardware beats renting frontier intelligence by the token. They are less compelling when the task calls for the strongest reasoning, the broadest world knowledge, fastest processing speed or the most polished coding performance available that demand large models and memory.

The Market Is Maturing

Running out of tokens can be a minor annoyance or a major inconvenience depending on the task. For business use, token cost and availability are starting to matter in the same way cloud-computing costs once became a central concern.

The industry has moved past the stage where AI could pose as near-infinite magic for a fixed monthly price. Vendors now have to price compute more openly and users have to manage context more deliberately. Buyers have to decide which jobs belong on subscriptions, which belong on usage billing and which belong on local machines. That is a healthier place for the market, even if it forces a more sober accounting of what AI delivers relative to what it costs. Why pay more when less good?