Matthew Sweeney, CPO and cofounder, Gomboc AI.
getty
One of the more interesting developments in recent months is not just that AI can generate code or detect vulnerabilities, but that it is starting to understand systems in a much deeper way. Projects like Mythos point in that direction. They are not simply scanning for known issues or patterns. They are reasoning across environments, mapping relationships between services, identifying dependencies and surfacing how different components interact. In many ways, they are beginning to approximate how experienced engineers think about architecture.
That is a meaningful shift, but it also exposes something uncomfortable.
For a long time, there has been an implicit assumption in engineering that the people building and operating systems understand them well enough to make informed decisions. That understanding has never been perfect, but it has been sufficient. Documentation, diagrams and dashboards have served as approximations of reality, and teams have relied on a combination of knowledge and experience to fill in the gaps.
As systems have scaled, that assumption has quietly broken down.
Modern cloud environments are not designed as cohesive systems. They are assembled over time through a mix of infrastructure as code, manual changes, third-party integrations and evolving requirements. Different teams contribute different pieces, often with varying standards and levels of discipline. Over time, the gap between what the system is supposed to be and what it actually is tends to grow.
Most teams are aware of this at a high level, but few have a complete, up-to-date understanding of how everything fits together in practice.
This is where AI changes the equation. When a system can analyze large volumes of configuration data, logs and dependency graphs simultaneously, it can surface relationships that are difficult for humans to track. It can identify indirect dependencies, highlight unexpected interactions and reveal how changes in one part of the system might impact another. In some cases, it can produce a more accurate and comprehensive view of the environment than the engineers responsible for maintaining it.
At first glance, that sounds like progress. In many ways, it is. Greater visibility helps teams identify risks earlier and understand the true shape of their systems.
But visibility alone does not solve the problem. In fact, it can make a different problem more obvious.
Understanding a system is not the same as being able to safely change it. AI can map relationships and highlight potential issues, but it cannot guarantee that a change will behave as expected when applied in a live environment. Engineering systems operate within constraints that are often implicit. There are dependencies that are not fully documented, behaviors that only emerge under specific conditions, and interactions that are understood through experience rather than formal definition.
When humans make changes, they rely on that context. When AI suggests changes, that context has to be inferred, and that introduces uncertainty.
As a result, teams are left in a familiar position. They may have better insight into their systems than before, but they still need to validate every action before it can be trusted. The increased visibility does not eliminate the need for careful review. If anything, it increases the volume of decisions that need to be made.
This creates a subtle but important shift. The challenge is no longer just understanding the system, but deciding what to do with that understanding at scale.
If AI can map your system more completely than you can, it raises a fundamental question about control. Who is actually responsible for how the system evolves?
If the answer remains the engineering team, then the system itself needs to be structured in a way that makes that control possible. That means reducing ambiguity, enforcing consistency and ensuring that changes can be applied in predictable ways. Without that structure, insight alone does not translate into better outcomes.
If, over time, some of that control shifts toward automated systems, the requirements become even stricter. It is not enough for a system to be understood. It needs to be operated safely without relying on constant human interpretation. That requires a level of consistency and constraint that most environments do not currently have.
What we are seeing today is a growing mismatch. AI is becoming better at understanding systems, while the systems themselves remain fragmented, inconsistent and difficult to operate with confidence. That gap is not just a technical issue; it is an operational one.
The organizations that benefit most from these advances will not be the ones that simply adopt more AI tools. They will be the ones that invest in making their systems more understandable in the first place. That includes standardizing how infrastructure is defined, reducing variation across environments and ensuring that policies are enforced as part of how systems operate rather than as an afterthought.
It also requires a shift in how teams think about ownership. Even if AI can provide a more complete map of the system, engineers still need to be able to reason about the implications of change. That means maintaining clarity in system design and discipline in how changes are introduced.
There is real value in having systems that can surface insights at a scale humans cannot match. But insight without reliable execution does not reduce risk. It simply makes that risk easier to see.
The more clearly a system is understood, the more visible its weaknesses become.
The goal is not to build systems that AI can understand better than we do. It is to build systems that are structured well enough that both humans and machines can operate them safely. Until that happens, better visibility will not translate into better outcomes. It will simply make the gap harder to ignore.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。