Reworking Architecture

Daniel McCullough

The Architecture Lens. Most enterprise AI discussions still happen at the wrong level. They focus on models, copilots, agents, and use cases. Those matter, but they are no longer the hard part. The harder issue now is architecture. Not just technical architecture, but the architecture of the organization, the architecture of the AI layer itself, and the architecture of the enterprise technology stack underneath it. Companies moving fastest are not simply choosing better tools. They are making more coherent choices across all three. That is where durable advantage is beginning to take shape.

The first architecture is organizational. Every enterprise is now wrestling with the same question: how much AI innovation should sit at the center, and how much should live in the businesses? A centralized model brings focus, standards, leverage, and a better chance of scaling what works. A federated model brings energy, proximity to the customer, and a sharper understanding of domain problems worth solving. Taken too far, each one fails in predictable ways. Let a thousand flowers bloom and you often get enthusiasm, scattered experimentation, and dozens of local proofs of concept that never move the economics of the company. Centralize everything and you may gain discipline, but you also risk slowing the organization down and suppressing the initiative that makes adoption real. The answer is not to declare one model right and the other wrong. It is to define the boundary between them with much more precision. What should be common across the enterprise? What should be reusable? Where do standards matter? And where do you want local teams pushing the frontier because they are closest to the workflow, customer, or decision that actually needs to change? The companies pulling ahead are not eliminating this tension. They are managing it deliberately.

The second architecture is of the AI layer itself. Here too, many companies are still asking shallow questions. They debate open source versus closed source as though it were a matter of philosophy. They debate optionality as though it were free. They debate model choice as though the decision ends on day one. In practice, the real issue is not which model looks best in a benchmark this quarter. It is what operating burden you are taking on over time. Optionality sounds attractive until it creates engineering drag, fragmented tooling, duplicated evaluation harnesses, and constant retuning. Closed systems can deliver speed, but they also concentrate dependency. Open systems can create flexibility, but they often shift more integration, governance, and optimization burden back onto the enterprise.

Then there is the part many companies still underestimate: day two cost. Inference cost is only one component. Re-engineering around model changes, managing versioning, revalidating outputs, tightening governance, and carrying the talent needed to keep the system reliable at scale are often the larger issue. Over time, the architecture question becomes less about raw model capability and more about how much volatility, cost, and dependency the enterprise is willing to absorb in exchange for speed. That is a strategic choice, not a technical one.

The third architecture is of the enterprise technology stack. This is where the conversation becomes more consequential than many leaders realize. If AI becomes the interface layer, the orchestration layer, or the control plane through which work increasingly gets done, then the logic of the underlying stack starts to change. Systems of record will remain. ERP, CRM, core platforms, industry systems, and data infrastructure are not disappearing. But their role may shift. For years, companies invested heavily in embedding process logic, workflow, and user experience inside those systems. If AI increasingly becomes the system of engagement sitting above them, then a different question emerges: how much capital should still be allocated to heavyweight systems designed around yesterday’s interaction model, and how much should be directed toward a more modular stack beneath a smarter interface layer? That does not mean ripping out core systems. It does mean rethinking what you want those systems to do, and what you no longer need them to do. The strategic issue is no longer just which system you standardize on. It is who controls the layer that coordinates work across systems, and therefore who owns the emerging operating logic of the company.

An integrated design problem. What makes this difficult is that the three architectures do not move independently. A federated organizational model tends to create pressure for more architectural flexibility. A highly centralized model often pushes toward fewer platforms, tighter standards, and more control over tooling. A decision to preserve broad model optionality may increase cost and complexity in ways the operating model cannot support. A decision to let AI sit above legacy systems may create a more attractive future interface, while also forcing uncomfortable questions about sunk cost and the pace at which core modernization should continue. Enterprises are not making isolated AI decisions. They are making interlocking choices about governance, economics, engineering, and capital allocation. Many of the frustrations companies are now experiencing come from treating these as separate conversations when they are, in fact, one integrated design problem.

This is also why so many AI efforts feel simultaneously impressive and unsatisfying. The demos are better than ever. Narrow use cases can produce real gains. Yet the broader enterprise still struggles to absorb what the technology now makes possible. The problem is rarely that the models are not good enough. More often, the organization has not yet decided how it wants to innovate, what tradeoffs it is willing to make in the AI layer, and how far it is prepared to rethink the role of the underlying stack. Until those decisions become more coherent, many firms will continue to accumulate pilots, tools, and fragmented wins without achieving a true shift in operating performance.

Creating durable advantage. This is now the central leadership challenge of enterprise AI. Not whether to adopt it. Not whether it matters. Those questions are largely settled. The real question is whether leadership teams can make disciplined choices across these three architectures before the market makes those choices for them. The winners will not necessarily be the firms with the most experimentation, the largest model budgets, or the most pilots in flight. They will be the ones whose organizational design, AI architecture, and stack strategy fit together in a way that produces durable advantage.

In the next phase of enterprise AI, coherence may matter more than brilliance. And architecture, in the broadest sense of the word, is where that coherence gets built.