Agents don't scale because the gap between "demo that works" and "system that handles real users doing unpredictable things" is fundamentally an engineering problem, not an AI problem. The LLM is the easy part. The hard parts are: deterministic guardrails around non-deterministic outputs, enterprise data integration (90%+ of which is unstructured and inaccessible), and the orchestration layer that decides which agent does what — and what happens when one fails mid-chain.
You're not missing a conceptual piece. You're likely underestimating the infrastructure tax of each scaling dimension.
[Source 2] nails this — the moment you put an LLM in front of real users, the problem changes entirely:
"consumers do crazy things right so you start to have to say well am I am I putting the LLM right in front of the consumer and if you are at that point then you need to guard rail it and that could be things like guard models it could be running you know deterministic flows in conjunction with the AI to keep it on track" — IBM Technology — "AI agents in 2025: Why agentic commerce isn't ready for Black Friday yet"
The fix most teams reach for: a planner layer that constrains the LLM to a pre-approved execution plan. Claude Code, Cursor, Windsurf — all of them do this. The agent doesn't freestyle; it proposes a plan, then executes within it.
[Source 3] states the actual number:
"less than 1% of enterprise data makes its way into generative AI projects today" — IBM Technology — "Unlocking Smarter AI Agents with Unstructured Data, RAG & Vector Databases"
90%+ of enterprise data is unstructured — contracts, PDFs, emails, transcripts. Your agent can reason perfectly and still give garbage answers because it can't access the data it needs. This is a data engineering problem, not a model problem. The pipeline to chunk, embed, govern, and serve unstructured data at scale is the bottleneck.
[Source 7] describes the real complexity:
"5 mini agents that then come back and aggregate and be able to surface whatever that actual output is" — IBM — "Using AI agents to transform your business at scale"
The question isn't "can I build one agent" — it's what happens when agent A calls agent B which calls agent C, and agent B hallucinates. Error propagation in multi-agent chains is multiplicative. Each agent has a failure rate; chain 5 together and your reliability drops to 0.95^5 = 0.77 at best. You need:
[Source 9] calls this out explicitly:
"our enterprise-specific data, our datasets... is not represented in these LLMs, so we need to go infuse those LLMs, those large language models, with our enterprise-specific data, fine-tune them, and tailor them to our usage" — IBM — "AI agents in action: From pilots to outcomes at scale"
Day one, the agent knows nothing about your business. Fine-tuning is expensive and slow. RAG is cheaper but requires the data pipeline from wall #2. Most companies stall here — the agent works on public knowledge but fails on internal processes.
[Source 9] again:
"You need to have enough instrumentation so you know where they're doing what kind of workflows and how do you course correct. How do you know that they're getting the right answers?" — IBM — "AI agents in action: From pilots to outcomes at scale"
Traditional APM (Datadog, Grafana) monitors latency and errors. Agent monitoring needs to track decision quality — did the agent pick the right tool? Did the plan make sense? Was the output factually correct? This observability layer barely exists as tooling today.
┌─────────────────────────────────────────────────┐
│ USER REQUEST │
└──────────────────────┬──────────────────────────┘
│
▼
┌──────────────────────────────────────────────────┐
│ PLANNER / ROUTER │
│ - Decomposes into sub-tasks │
│ - Selects which specialist agents to invoke │
│ - Defines deterministic guardrails per step │
└──────────────────────┬───────────────────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Agent A │ │ Agent B │ │ Agent C │
│ (domain │ │ (domain │ │ (domain │
│ expert) │ │ expert) │ │ expert) │
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│VALIDATOR│ │VALIDATOR│ │VALIDATOR│ ← deterministic check
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
└────────────┼────────────┘
▼
┌──────────────────────────────────────────────────┐
│ AGGREGATOR / VERIFIER │
│ - Merges outputs │
│ - Checks for contradictions │
│ - Human-in-the-loop for high-risk decisions │
└──────────────────────┬───────────────────────────┘
▼
┌──────────────────────────────────────────────────┐
│ OBSERVABILITY / FEEDBACK LOOP │
│ - Decision audit trail │
│ - Quality scoring per agent │
│ - Drift detection │
└──────────────────────────────────────────────────┘
| Gap | Why It Matters | Most Teams Miss It Because... |
|---|---|---|
| Inter-agent validation | One bad agent poisons the chain | They test agents individually, not as a pipeline |
| Unstructured data pipeline | 90% of useful data is locked away [Source 3] | They assume "just add RAG" solves it |
| Agent registry / discovery | At scale, agents need to find each other | Works fine with 3 agents, breaks at 30 |
| Decision-quality monitoring | Latency metrics don't tell you if the answer was right | Traditional APM doesn't cover this |
| Graceful degradation | What happens when the LLM provider is down or slow? | Happy-path thinking |
| Cost at scale | 1000 users × 5 agent hops × ~$0.03/call = $150/day minimum | Demo costs ≠ production costs |
| Fix | Effort | Impact | Notes |
|---|---|---|---|
| Add planner layer with deterministic guardrails | 2-3 hours | High | Prevents the "consumer does crazy things" failure mode |
| Build unstructured data pipeline (chunk + embed + serve) | 1-2 days | High | Without this, agents answer from vibes not data |
| Add validator between each agent hop | 30 min per agent | High | Catches error propagation before it compounds |
| Instrument decision-quality metrics | 1 day | Medium | You can't improve what you can't measure |
| Build agent registry with capability declarations | 2-3 hours | Medium | Only matters once you have >5 agents |
| Add cost tracking per request | 30 min | Low-Medium | Prevents bill shock at scale |
The sources consistently point to the same conclusion: the model is not the bottleneck, the infrastructure around the model is. Scaling agents is a systems engineering problem — data pipelines, orchestration, validation, observability, and cost management. The teams that treat "agent" as an AI problem instead of a distributed systems problem are the ones that stall at the pilot stage.
The thing most people miss: you need deterministic systems wrapping non-deterministic ones, not the other way around. The LLM proposes; deterministic code disposes.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。