Recently, a colleague of mine was working on a mathematical model in Excel. He asked Copilot to solve a complex problem. The answer the spreadsheet produced wasn’t quite right. He asked Copilot what it did to figure out the answer. It started spitting out Python code. 

My colleague was not a developer. He had never written a line of code in their life. He had no interest in becoming a developer. However, at that moment, he had become one. 

AI An Abstraction Layer 

When it comes to software development, I’ve referred to AI as an “abstraction layer”. My data scientist friends hate when I say that, but the reality is that we’ve been following a natural trend in programming abstraction since the dawn of Babbage’s Difference engine: 

• We started with binary. A handful of people could effectively program in ones and zeroes. 

• Next, we moved to assembly language. This opened the doors to more complex programs and more potential programmers. 

• Soon after, we created higher level languages. This included C, C++, Fortran, Cobol, etc. These lead to vastly more complex programs and made professional programming a viable career. 

• Eventually, we created simplified languages like Visual Basic and low-code. This made it easier to create visual representations of programs and logic. It also opened the doors to citizen developers. 

• Now, we have AI. Anyone can prompt to create software – including that pizza place up the street. 

At each level, the abstraction layer got further and further between the programmer and the underlying binary. At the same time, the gamut of who could become a “programmer” got larger. We’re now at a point where the abstraction layer is so great that programmers no longer consistently identify themselves as programmers. 

Put another way, we no longer have “professional” and “citizen” developer – we just have developers. And some of them are accidental developers – they don’t even know they’re writing code. 

The Fundamental Flaw With Tools Creating Tools 

My colleague’s interaction with AI is not a unique one. I’ve seen this with others as well, and we’ve posited the “tools creating tools” space for some time when it comes to AI. 

However, we’ve glossed over a significant challenge: In the more than a century we’ve been perfecting software engineering, programmers have focused on writing secure, reliable, and redundant code at scale. All of the steps of the traditional software development lifecycle (SDLC): analyze/plan, design, build/maintain, test, and deliver were formalized under the assumption that humans manage each stage. 

In my conversations with accidental developers, however, they’ve handed off the wheel of these stages to AI – if they know there are stages at all. When I ask them if they’ve reviewed the code generated, they rarely do. If they do, they often don’t understand it. Some have the wherewithal to ask AI to test the code generated, but that’s an explicit ask and often done with the same AI that wrote the code (a situation that would be frowned upon with human developers). Delivery has its own challenge: “it ran on my laptop” takes on entirely new meaning when AI has installed packages and a container runtime on your machine you didn’t install yourself and now need to replicate in the cloud. And proper analyzing/planning and designing beforehand? Forget about it. 

This hasn’t been helped by the fact that we’ve seen a shift in coding agents grow in capabilities to do multiple parts of the SDLC. We’ve gone from multiple agents from different vendors communicating intent across multiple phases of the SDLC, to single agents doing everything. Separation of duties, this is not. 

We Need To Address Developer Safeguards Tactically And Strategically 

It would be silly to presume we can close Pandora’s Box at this point. Now that the agentic software development genie is out of the bottle, we can’t (and shouldn’t) tell people, “Don’t write code”. Two reasons: 1) Fundamentally, programming should be open to all and 2) As made clear in this blog, some people don’t even know they’re writing code to begin with. This is especially true as tools creating tools cascades to multiple levels of hammers making other hammers. 

In short, we need to solve this tactically and strategically: 

• You can’t hallucinate security, reliability, and redundancy. This means, tactically, we need to educate and train users. They need to know that just as AI can hallucinate answers you have to double-check, it can create code that you also have to double-check. Users must learn to prompt to test software created – even if they’re unsure software was created in the first place. They must learn the disciplines of properly analyzing/planning and designing before they even start prompting. They must be taught the difference in deploying a prototype locally versus deploying an enterprise-grade system in the cloud. Even in smaller situations, like when Excel creates Python, users must be trained to test code created and verify results.

• In the future, the models themselves will become the responsible safeguards. More strategically, just as safeguards are being built into AI models to protect against personal (e.g., conversations about suicide) and societal (e.g., questions about warfare) threats, equivalent safeguards must be created for software development. Going forward, models must be trained so that every time code is generated by AI – whether the user knows about it or not – it’s tested for security, reliability, and redundancy. This should be done preferably by different agents. Delivery plans must be created for multiple environments varying in scale, and users must be informed about the blast radius of their actions.

Spec-driven development practices will help this, but there are fundamental requirements that must be built into the models themselves. The onus is on users and model creators to work together and build software securely with AI, whether they call themselves developers or not.

Forrester has a full team of analysts covering the revolution of agentic software development and the dawn of the accidental developer – I am the team’s Reesearch Director. Schedule a guidance session with us if you’re a client to discuss the ramifications of this, or leverage our Forrester AI for instant insight. 

Categories

Blog

Announcing The Forrester Wave™ On Extended Detection And Response Platforms: Platformization, AI, And … AI

Last week, Forrester released The Forrester Wave™: Extended Detection And Response Platforms, Q2 2026. This is the third iteration of the extended detection and response (XDR) Wave, with prior versions published in 2021 and 2024. This Wave differs significantly from the past, especially because of: The number of vendors. This year, only seven vendors were […]

Blog

The Cost Of AI Productivity Is Less Creativity

AI is now standard across marketing agencies — but its impact remains constrained. Forrester’s “The State Of AI Inside US Marketing Agencies, 2026” finds that CMOs must shift their AI ambitions toward creativity, differentiation, and growth to unlock real value.