Khurram Mir is the founder of Kualitatem and Kualitee, focused on software quality engineering, product development & tech entrepreneurship.
getty
For the past three years, I've watched a pattern repeat itself across software engineering organizations—from early-stage startups to publicly traded software companies. The team adopts an AI coding tool. Velocity climbs, leadership celebrates and QA headcount gets quietly frozen or reduced. Then, six to nine months later, a production incident exposes a logic error that looked perfectly reasonable, passed review and sailed through the rest of the suite.
GitHub research found that AI-assisted developers ship code up to 55% faster. However, output speed and output reliability aren't the same measurement, and most organizations are only tracking one of them.
Those cutting QA investment argue that AI writes cleaner code than rushed humans, so there's less to catch. That argument misunderstands what AI actually does.
The failure mode here isn't obvious, and that's precisely what makes it dangerous.
AI code generators are pattern completion engines. They're extraordinarily good at producing code that resembles correct code. They're not reasoning about your business logic, your edge cases or the system-level assumptions that a developer—who has since left—baked into your architecture three years ago. The output looks clean because it's syntactically fluent, not because it's contextually accurate.
The practical consequence is what I call the "looks right" problem. In code reviews, there's social pressure to approve. When AI-generated code arrives formatted, readable and confident, the bar for pushback rises.
Engineers accept suggestions they haven't fully traced because questioning them feels like slowing things down. A study on technical debt formation published in Information and Software Technology pointed to review quality degradation, not volume of bad code, as the primary driver of compounding defects.
There's also a structural problem underneath the clean surface. AI suggestions frequently introduce:
• Undocumented Dependencies: Third-party libraries or internal APIs pulled in without explicit justification or ownership.
• Edge-Case Assumptions: Implicit handling of null values, boundary conditions or data formats that were never discussed or specified.
• Architectural Shortcuts: Patterns that work in isolation but conflict with the broader system design in ways that only emerge at runtime.
The cleaner the output looks on the surface, the more dangerous the blind spot underneath.
Most QA processes were designed around a simple premise: The developer who wrote the code understands it. That premise no longer holds.
When AI generates a function, the developer may have reviewed it at a surface level without tracing through its full behavior. When the same AI (or a similar one) is then used to generate tests for that function, you don't have quality assurance. You have circular validation. The model that produced the code and the model checking it share the same statistical tendencies, the same training blind spots and the same confidence in plausible-looking outputs.
The real-world parallel is editing your own writing. Every editor knows you can't reliably proofread your own work—not because you're careless but because your brain fills in what it expected to write rather than reading what's actually there.
"AI testing AI" functions the same way. It's an echo chamber dressed up as a review process.
The most effective engineering organizations that I've seen navigate this well share a common structural decision: They've separated code generation from test design. AI handles the execution layer. Humans own the test strategy.
In practice, this means QA engineers are no longer just writing test cases. They're asking interpretive questions:
• What was this code supposed to do? This traces intent, not just syntax.
• What assumption is it making about the data it receives? This surfaces implicit contracts between system components.
• Where does this behavior break down at the edges? This tests boundary conditions that AI-generated code is statistically likely to underhandle.
These teams have also shifted their investment from unit test volume toward behavioral and integration testing.
Unit tests confirm that individual functions execute. Integration tests confirm that systems behave as intended under real conditions. The distinction matters because AI-generated code often passes unit tests while introducing failures at the integration layer.
Additionally, several CTOs I know have formalized a policy of treating AI-generated pull requests with the same scrutiny as code submitted by an external vendor. That framing changes the review dynamic immediately. You wouldn't merge a third-party contribution without understanding it. The same standard should apply to a contribution from a model that has no understanding of your system.
The instinct to reduce QA investment as AI output scales is almost exactly backwards. More AI-generated code means more output that requires interpretive review. Human judgment about whether what was built matches what was needed requires experience.
The worst scenario I've seen unfold in practice is junior developers accepting AI output at face value, with no senior QA layer and teams racing to production on velocity metrics while defect density quietly climbs in the background. By the time it surfaces, it looks like a testing failure when it's actually a structural one.
This is the new technical debt. It doesn't show up in your sprint metrics or trigger alerts. It accumulates until a production incident forces the conversation nobody wanted to have at scale.
The teams that will earn trust in the AI era aren't the ones shipping fastest. They're the ones whose AI-assisted output actually holds up when customers use it.
Here's the question I'd leave every technology leader with: Does your testing discipline scale as quickly as your AI adoption does? If you can't answer that confidently, the gap between those two curves is exactly where your next crisis is forming.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。