If projections about the rapid growth of the agentic AI software market are to be believed, the typical enterprise will soon be devoting significant shares of its total AI budget to paying for AI agents -- meaning tools that can perform actions within digital systems using AI.
But whether all of those AI agents will actually create value depends, in large part, on how effectively businesses manage their agentic AI costs. AI agents deployed inefficiently risk driving AI spending through the roof without commensurate boosts in productivity or operational efficiency.
A key question facing IT leaders, then, is how to control AI agent costs before they spiral out of control -- and it's a question CIOs need to begin answering now, while businesses remain in the early stages of agentic AI adoption and still exercise significant control over how they implement and manage AI agents.
Related:Will the music stop for AI's funding dance?
Broadly speaking, AI agent spending breaks down into four categories:
The price of agentic software. While some agents are free of cost (indeed, a growing collection of free, open source AI agents is available), most enterprise-ready agents cost money. Pricing models vary; some agents are available via a one-time payment, while others come with recurring subscription fees, and still others are priced based on usage.
Token costs. When agents interact with LLMs, they typically incur a token cost. Unless this fee is built into the agentic software platform (which is usually only the case under usage-based pricing models), businesses must pay for it separately. The more frequently agents send data to LLMs and the more complex the requests are, the higher the token costs. (Token costs typically apply for only businesses that use third-party models -- but if you operate your own, in-house model, you still have to pay for the energy costs of each model query.)
Infrastructure costs. Like any type of software workload, AI agents require infrastructure to host them -- so businesses must pay for the compute and memory resources that agents consume when they operate.
IT management costs. Also, like most types of software, agents must be monitored, secured, updated and so on. These operations require IT resources, including staffing and tools.
Of those four categories, only one -- the cost of agentic AI software -- is relatively predictable and easy to control. Agentic AI software vendors are usually transparent about their pricing, making it easy enough to anticipate how much you'll pay for the software itself.
Related:The hidden high cost of training AI on AI
Managing agentic AI costs across the other three categories, however, tends to be challenging. The core reason is that AI agents can behave in ways that are difficult to predict. This is because modern AI systems are, by design, non-deterministic -- meaning the same input will not always yield the same output.
For AI agents, non-determinism has the effect of making it virtually impossible to anticipate exactly how an agent will fulfill a request -- or even to assume that the way it completed a task historically will continue to be the way it does so in the future. By extension, token costs, infrastructure resource consumption rates and agent maintenance requirements may also vary.
To place this challenge in context, let's look at how the costs of real-world agentic AI processes can vary depending on how agents approach a task.
Imagine a software development agent tasked with generating code to implement a new button inside an application. There is no way to know in advance exactly which code the agent will produce. Nor is it possible to predict precisely how it will go about testing and debugging its code. Yet the total lines of code it produces and the total number of interactions it has with LLMs while writing and validating the code have a significant impact on the total cost of the process.
Related:Red Hat CIO Marco Bill: Resource control is key for AI sovereignty
As another example, take a content production agent that a marketer uses to create a product brochure. Here again, it's impossible to know how much text or how many images the agent will generate, how many times it will ask LLMs to reference the business's existing product brochures for context, or how many iterations of the new brochure it will work through before producing a final product. More work by the agent translates to higher costs, due mainly to token usage and CPU and memory overhead. It may also increase the time and effort the IT department needs to devote to managing agents, since more active agents require greater oversight and maintenance.
It's possible for humans who deploy AI agents to define parameters (e.g., "keep total lines of new code below 100" or "look at only the three most recent product brochures as examples") that limit the agents' range of action -- and, by extension, the costs they incur.
The problem with doing so, though, is that it undercuts part of the value of using AI agents in the first place. The more time users have to spend telling AI agents exactly how to go about completing tasks, the less time and mental load the agents save for humans. In addition, restricting the length or complexity of the work that AI agents produce may have the effect of reducing its quality.
Hence the need for businesses to find ways to leverage AI agents' full potential, but without breaking the bank.
Fortunately, there are ways to control agent costs without setting artificial or arbitrary limits on agents' ability to act. Business and IT leaders should consider the following:
Choosing flexible agentic AI platforms. When procuring agentic AI software (or building it in-house, if you opt for that approach), prioritize products that offer flexible configurations. The more freedom the business has over where its agents are hosted, which LLMs they use and how they are managed, the easier it will be to manage costs.
Considering low-cost LLMs for low-stakes agents. Generally speaking, the better the LLM (meaning those capable of generating more complex or accurate results), the more it charges per query. Not all agents need the best LLMs; businesses can save money by configuring agents to interact with lower-cost LLMs when the tasks they're charged with are less complex or require lower levels of accuracy.
Using LLMs to predict the costs of agentic workflows. It's possible for agents to describe how they plan to carry out a task before they actually execute on it. Reviewing the plan is a way to predict how much it is likely to cost in terms of tokens and resource usage -- and while it's not practical to have a human review every proposed workflow, LLMs could be deployed to automate cost estimates. The review process comes with its own costs (because it requires sending the review request to an LLM), but it may save money overall if it prompts agents to find a new, lower-cost way to execute a task.
Tracking the actual costs of agentic workflows. In addition to predicting costs beforehand, businesses should monitor the actual cost incurred by each AI agent for every task it completes. Some agentic AI platforms offer built-in cost-monitoring capabilities; alternatively, monitoring total tokens used and their associated costs provides valuable insight.
Optimizing cost-effective agentic workflows. If businesses track the cost of agentic workflows, they can also assess and correct cost-inefficiencies (such as an agent evaluating content that is non-essential).
Repeating cost-effective workflows. Going a step further, organizations can identify agentic workflows that are particularly cost-effective, then configure agents to follow the same or similar processes when possible. This results in something akin to a "prompt library" -- except instead of validated AI model prompts, it contains approved agentic workflows.
Caching data and content. If agents repeatedly request similar data or generate similar content, it may be possible to save money without compromising quality by caching the data or content. In other words, rather than requiring an agent to send the same type of query to an LLM repeatedly, it could cache the query results and reference them -- reducing token usage.
Setting token quotas. To guard against situations where a buggy or out-of-control AI agent runs up a very large bill, organizations can set quotas that restrict how many queries the agent can submit per request or within a specified time period. In general, these limits should be high to ensure that agents are able to complete tasks; nonetheless, having some hard-coded upper-limits is important for preventing high spending under unusual circumstances.
Avoiding unnecessary agent deployments. More AI agents are not necessarily better, certainly not from a cost-management perspective. To avoid unnecessary spending, businesses should review the agents they currently have deployed and ensure that each one is actually warranted and useful -- a practice similar to the control of SaaS sprawl.
Of all those practices, choosing an agentic AI platform and architecture that maximizes the ability to control costs is the most important step most businesses should take early on to get ahead of unnecessary agentic AI spending. Implementing cost monitoring for AI agents early on is also critical, since it's impossible to rein in costs if you don't know what they actually are.
From there, businesses can implement more tactical practices, such as content caching and automated workflow repetition, to reduce agent costs on a day-to-day basis.
It's also important to complement technical controls with organizational responsibilities and processes for agentic spending management. For instance, a business might require that anyone who deploys an AI agent assess the agent's total costs before doing so. Periodic, recurring reviews of agentic AI spending and cost-optimization opportunities can also go a long way toward helping keep financial waste in check.
The characteristics that make AI agents so powerful -- their ability to act autonomously and flexibly -- also make their costs difficult to predict. But with creative strategies and controls, organizations can ensure the cost of AI agents doesn't outweigh the value they create.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。