Edith Harbaugh is the CEO and Co-founder of LaunchDarkly.
getty
For much of my product and engineering career, the hardest part was building. Gathering product requirements, working with teams to build and test the software to release to customers was my job. The speed at which we could build was the primary constraint. Everything else, whether it be process, tooling or cultural, was organized around building faster.
That has changed dramatically in the last few years, as AI has made software feel both cheaper and easier to create. The hard part has moved, and the problem facing us now is what comes after the software is built.
That shift is something I think most engineering leaders are still grappling with. Just as the shift to the cloud in the 2000s made dot releases obsolete in favor of software as a service, AI software development makes runtime control a necessity.
Engineering teams are now shipping code at a pace that would have been unimaginable even five years ago. But with AI, volume is only part of the problem. Engineers are regularly changing features they've already shipped—swapping models, tuning prompts, releasing new capabilities. While each change may be intentional, what emerges in production often isn't. As teams increasingly deploy agents in customer-facing products, the gap between software’s intent and its behavior has the potential to widen.
We've already seen what that looks like: unexpected outputs, runaway costs and behavior that degrades gradually and then suddenly. The issue isn't poor design at build time; it's the absence of any real capability to observe, intervene and correct issues in production.
The instinct is to respond with caution, keep AI projects in staging, run longer internal tests and wait until the team feels confident enough to ship. These are similar anti-patterns we’ve seen before about more and more staging tests. However, this caution is expensive, and more importantly, it doesn't actually buy confidence.
In 2015, I wrote, somewhat hyperbolically that “Staging Servers must Die!” Confidence doesn't come from waiting. It comes from having the ability to act when something goes wrong, and to be able to act extremely quickly. Teams should understand that being able to control in production allows you to ship quicker, with less risk.
The teams getting this right aren't theorizing about runtime control. They're practicing it daily. Changes go out incrementally, and exposure is controlled dynamically, targeted by user segment, risk tolerance or context. Behavior is monitored for the indicators that matter the most to their business, including uptime, quality, cost and meeting the intended behaviors.
When you can't fully predict what your software will do in production, the only rational way to move fast is to limit how much can go wrong at once, and have instrumented metrics to know when something has gone wrong.
The most forward-thinking teams are taking control a step further and not just tracking degradation, but adding automated responses. Guardrails detect degraded AI output and limit exposure automatically. Rollbacks are triggered by output quality signals in addition to error rates. The result is control that operates faster than any team can move manually, and that speed is what makes aggressive experimentation possible. You can move fast when you know failure won't compound before you catch it.
You can't eliminate failure; that’s not realistic in complex systems. Rather, the goal is to make failure non-disruptive, and that’s only possible when control lives in production.
If you’re running an engineering org right now, my question is: When something goes wrong in production—not a crash or major outage, but a subtle behavioral regression—how quickly can you detect and contain it, and how much customer impact is felt in between?
For most teams, the honest answer is "not fast enough." And that gap between when something goes wrong and when you can actually act, is where customer impact lives. The constraint has moved to production itself, to whether you have genuine control over systems that are changing constantly, often in ways nobody explicitly authorized. The teams that can close that gap will be faster because they can experiment aggressively without fearing regressions they can't contain.
Runtime control isn't the thing that slows innovation down. It's the thing that makes sustained innovation possible.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。