I Ran My AI Code Auditor on Its Own Source Code

Auditing the Auditor

I spent the last day pointing DriftCode Auditor at its own source code — both the free version and the Pro version.

I wanted to see what the tool would actually surface when aimed at a real, non-trivial codebase that had been written with heavy AI assistance. That codebase happens to be the tool itself.

Why bother

I’ve been using Grok Code Fast 1 a lot while building this. Like most people vibe coding, I’ve noticed the same recurring patterns: overly broad exception handling, duplicated scaffolding logic, functions that look complete but quietly skip error cases or edge conditions.

The whole point of the tool is to catch exactly those things.

I did full scans on both the public free repo and the private Pro repo using --privacy --maintainability. No exclusions for the rules themselves. I wanted to see the raw output.

What showed up

The two most consistent problems were exactly what I expected:

• Broad except Exception blocks that swallow errors and return early. These appeared in the scanner and in the config loader. Same pattern in both the free and Pro codebases. The comments were usually some version of “gracefully handle.”

• In the Pro rules, the detection logic for placeholder functions and duplicate logic was using nearly identical crude heuristics in multiple places. One rule would look for function definitions, another rule a bit further down would do almost the same check with slightly different window sizes. The tool correctly identified this as both placeholder_stub and duplicate_logic.

There were also the usual long functions and a few self-referential noise hits (the TODO detection rule flagging its own comments, for example). Some of the Pro rules were noisy when pointed at their own source, which wasn’t surprising.

The fixes

I fixed the two clearest problems:

• In both scanner.py and config.py (free and Pro), I replaced the blanket except Exception with more specific exception handling and a deliberate re-raise for anything unexpected.

• In the Pro reporter, I extracted the duplicated issue formatting logic into a small helper function so the same block wasn’t repeated in the full and truncated report paths.

I left other things alone for now. Some of the long functions are still long. The Pro rules still generate some self-referential noise on their own code. I’m not trying to make the tool look perfect — I’m trying to make it actually better.

What this showed me

The main takeaway wasn’t surprising, but it was useful to see it in my own code: when you generate a lot of code with AI and move quickly, the same small set of mistakes appears repeatedly. Broad exception handling and duplicated logic were the most obvious ones here.

Running the tool on its own source also made the limitations of some of the Pro rules more visible. That’s useful feedback for future improvements.

This isn’t a story about some profound moment of self-awareness. It’s just the result of actually using the thing I built on the code I built it to protect.

Current state

The fixes are in. Free version is now at 0.2.2 and Pro is at 0.2.3.

If you’re generating a lot of code with Cursor, Claude, or similar tools right now, I’d be interested in what the current version surfaces on your actual work. The reports are still more useful than flattering, which is the point.

The free version is on PyPI. The Pro rules are available through GitHub Sponsors.

https://github.com/DriftApplied