Every company is shipping AI features right now. RAG pipelines, agent workflows, LLM-powered support bots, code assistants, content generators. The race to production is real.

But here is what nobody talks about at the architecture review: your observability bill is about to explode.

Traditional microservices generate predictable telemetry. A request comes in, hits a few services, returns a response. You trace it, you log it, you metric it. Done.

AI workloads are fundamentally different. And if you are monitoring them with the same tools and the same approach, you are about to get a very unpleasant invoice.

The telemetry explosion nobody planned for

A single LLM inference request can generate:

• Prompt and completion tokens - often thousands per request, each needing tracking for cost attribution
• Embedding vectors - high-dimensional data that traditional log systems were never designed to store
• Chain-of-thought traces - multi-step reasoning that creates deep, branching trace trees
• Evaluation metrics - accuracy, hallucination rates, relevance scores that do not map to HTTP status codes
• Retry and fallback logs - model timeouts, rate limits, and graceful degradation events

A typical RAG pipeline hitting a vector database, retrieving context, calling an LLM, and post-processing the response generates 10-50x more telemetry data than an equivalent traditional API call.

Grafana's 2026 Observability Survey - the largest community-driven survey in the space, with over 1,300 respondents - found that 92% of practitioners see value in using AI within observability. But buried in that optimism is a cost problem that is already hitting teams hard.

Where traditional observability tools fail

Here is the core issue: tools like Datadog, New Relic, and Splunk price by data volume. Logs per GB. Custom metrics per host. Traces per span. APM per host.

That pricing model was designed for a world where telemetry volume scaled roughly linearly with traffic. More users meant proportionally more logs, more traces, more metrics.

AI workloads break that assumption completely.

The token economy creates a new cost dimension

Every LLM call needs token-level cost attribution. You need to know:

• Cost per request, per user, per feature
• Which prompts are burning budget (a poorly tuned system prompt can 10x your token usage)
• Whether your caching layer is actually working
• Cost-performance tradeoffs between models (is GPT-4 class accuracy worth 10x the cost of a smaller model for this use case?)

Traditional APM tools treat all this as "custom metrics" - and charge accordingly.

Evaluation metrics do not fit existing models

In traditional systems, "is it working?" means: is the HTTP status 200? Is latency under the SLO? Is the error rate below threshold?

For AI workloads, "is it working?" means something entirely different:

• Is the model hallucinating more than last week?
• Are responses relevant to user queries?
• Is the model drifting from its training baseline?
• Are embeddings still clustering correctly?

These semantic quality metrics generate continuous streams of evaluation data. Ship that to Datadog as custom metrics and watch your bill double overnight.

Traces become trees become forests

A simple chatbot interaction might look like this under the hood:

1. User message received
2. Intent classification (LLM call #1)
3. Context retrieval from vector DB
4. Document ranking and filtering
5. Prompt assembly with retrieved context
6. LLM completion (LLM call #2)
7. Response validation (LLM call #3)
8. Safety check (LLM call #4)
9. Response delivered

That is 4 LLM calls, a vector DB query, and multiple processing steps - for a single user message. Each step generates spans, logs, and metrics. Your trace storage just went up 4-8x compared to a traditional request-response flow.

The real numbers

Let us do the math on a mid-sized AI deployment:

Scenario: Customer support bot handling 10,000 conversations per day, averaging 5 turns each.

• 50,000 user messages per day
• 4 LLM calls per message = 200,000 LLM invocations
• Average 2,000 tokens per invocation = 400 million tokens per day
• Each invocation generates ~5 trace spans = 1 million spans per day
• Token tracking, evaluation, and cost metrics = ~20 custom metrics per invocation = 4 million metric data points per day
• Logs for prompts, completions, and debugging = ~2KB per invocation = 400 MB of logs per day

On Datadog's pricing:

• APM: $31/host/month (but you need dedicated GPU hosts)
• Log Management: $0.10/GB ingested + $2.55/million log events
• Custom Metrics: $0.05 per custom metric per month (4M data points gets expensive fast)

Teams report that adding AI workload monitoring to their existing Datadog setup has increased their observability bill by 40-200%, depending on the volume and how many custom metrics they instrument.

Some teams have responded by simply not monitoring their AI workloads properly - which is worse.

What actually works

The observability industry is waking up to this. Here is what forward-thinking teams are doing:

1. Separate your AI telemetry pipeline

Do not shove AI telemetry into the same pipeline as your application telemetry. The data shapes are different, the retention requirements are different, and the query patterns are different.

AI observability data includes large text payloads (prompts and completions), high-cardinality token counts, and evaluation scores that need statistical analysis - not just threshold alerting.

2. Use OpenTelemetry for AI instrumentation

OpenTelemetry's semantic conventions for GenAI are maturing fast. The gen_ai.* attribute namespace provides standardized ways to capture:

• gen_ai.usage.input_tokens / gen_ai.usage.output_tokens
• gen_ai.request.model
• gen_ai.response.finish_reasons

This gives you vendor-neutral telemetry that does not lock you into any specific observability platform. If your bill spikes, you can switch backends without re-instrumenting.

3. Sample intelligently, not uniformly

Not every AI request needs full telemetry. A 10% sample of normal conversations gives you solid statistical coverage. But you want 100% capture of:

• Requests that hit cost thresholds
• Requests with low evaluation scores
• Requests that trigger safety filters
• Requests from high-value users or enterprise accounts

Head-based sampling does not work for AI workloads. You need tail-based sampling that makes decisions after seeing the full request lifecycle.

4. Consolidate your observability stack

This is the biggest lever. If you are paying Datadog for APM, a separate vendor for log management, another for traces, and now adding AI-specific tooling on top - you are burning money on integration overhead alone.

An open-source, self-hosted observability platform eliminates per-GB and per-metric pricing entirely. Your cost becomes compute and storage - which are predictable, controllable, and do not spike when you ship a new AI feature.

OneUptime handles monitoring, incident management, status pages, logs, APM, and traces in a single platform. Self-hosted, it is free. On SaaS, pricing is usage-based by GB ingested - not by the number of custom metrics, not by host count, not with surprise overage charges.

For AI workloads specifically, this means:

• Log all your prompts and completions without worrying about per-event pricing
• Track custom evaluation metrics without per-metric charges
• Store deep traces for every AI pipeline step without span-based billing
• Set up alerts on model drift, cost anomalies, and quality degradation using the same platform you already use for infrastructure monitoring

5. Build cost attribution into your pipeline from day one

Do not wait until the bill arrives to figure out where the money went. Instrument cost tracking at the application level:

# Tag every LLM call with cost context
span.set_attribute("ai.cost.input_tokens", input_tokens)
span.set_attribute("ai.cost.output_tokens", output_tokens)
span.set_attribute("ai.cost.model_price_per_1k", model_price)
span.set_attribute("ai.cost.estimated_usd", estimated_cost)
span.set_attribute("ai.feature", "customer_support_bot")
span.set_attribute("ai.customer_tier", customer.tier)

This lets you answer questions like "How much does our AI support bot cost per enterprise customer per month?" - which is the question your CFO will ask approximately three months after launch.

The bottom line

AI workloads are not just "more data." They are a fundamentally different telemetry profile that breaks the assumptions traditional observability pricing was built on.

The teams that figure this out early - who build cost-aware AI telemetry pipelines, who consolidate their observability stack, who use open standards and open-source tooling - are the ones who will actually be able to afford to run AI in production long-term.

The rest will discover that their monitoring bill has quietly become their second-largest infrastructure cost, right behind the GPU instances themselves.

Do not let your observability vendor be the biggest winner from your AI investment. Own your telemetry. Control your costs. Ship AI features without the bill shock.