Why would you worry about them? LLMs promise something close to infinite capability, and worrying about the meter feels like someone else's problem. Ship the feature. Let the model think as long as it needs to, right?

Right. Until the bill arrives. 

The shock

Picture it: your best chat assistant just launched. A few hundred test conversations, a bill you barely notice, a demo that gets the team excited. You ship it. Usage climbs the way usage does when a feature works well. Well done, you! More users, longer sessions, an agentic loop that now calls the model three or four times per user action instead of one. Nobody changes the code. 

Then someone opens the billing page expecting last month's number, and is met instead by a gargantuan figure. Sadly, this happens more often than we like to admit. 

A brief note on tokenomics

The surprising nature of these cost mishaps lies in the novelty of tokenomics. Ordinary cloud spending scales with the resources you provision: instances, database tiers, cores. LLM spending, instead, scales with the volatile shape of conversations. A verbose system prompt, a user who pastes in a long document, or a retry loop after a malformed response can multiply the cost of a single request without any code changes. The catch, sadly, is that you still pay for compute resources. Only now it’s compute resources that someone else's behaviour forces you to use.

The multiplier

Multiply that unpredictability across providers and brace for impact. You don’t like lock-in, so one feature runs on OpenAI, another on Anthropic. Someone moved a cost-sensitive batch job to a self-hosted model six months ago. Each of these has its own dashboard, billing cycle, and even its own definition of a "token." This lack of centralisation creates a crisis, because at no point can someone easily pinpoint the exact spending within a specific date range. 

To add fuel to the fire, cloud billing systems often tell you what you’ve spent only after you've spent it, with latency erring into weeks. By the time an alert fires, your requests have gone out, the provider has billed for them, and the only lever left is turning the whole feature off. 

Regaining control

LLM costs, at this point, are a risk. And they need to be treated as such. If we were to follow basic risk framework guidelines (which I encourage you to do), control requires three pillars: prevention, detection, and mitigation.

Empirically, good prevention and detection decrease your chances for mitigation

In a nutshell

The LLM ecosystem asks for an odd kind of double vision: enough hype to keep building, enough skepticism to ask what the invoice will look like. Responsibility, more than caution, is what holds the two together. Innovation has never been free, but as providers lean harder into tokenomic pricing, the cost stops being merely high and starts being unpredictable. Sadly, unpredictable is harder to plan around.

Staying in control of costs luckily doesn't require retraining your engineers as accountants. There are tools built for token cost management, our Otari among them, and the ecosystem around it is only going to get more tumultuous before it settles into a market standard.

If you take anything away from this post, I hope it is a clear requirement to plan and control. We don’t want you to become another horror story on social media feeds. If you’re curious about how you can use Otari to achieve this, give a read to our cost documentation.