Published May 26, 2026 | Version 1.0
Description
AI assistance has made source-code review cheap, and like every productivity multiplier in the history of engineering it has therefore made being wrong cheap. The open-source security community has spent the last eighteen months noticing the result: bug-bounty intake queues drowned in plausible-sounding but fabricated vulnerability reports, with the curl project's January 2026 closure of its HackerOne programme the headline example. The conversation so far has mostly been complaint. What is missing is a taxonomy of the failure modes, a pre-send filter that catches the most mechanical of them, and honest case studies from researchers who have themselves shipped the slop.
This paper supplies all three. We propose a four-class taxonomy (bug-shape fabrication, evidence fabrication, severity inflation, trivial-as-critical), present two real disclosure withdrawals and one near-miss caught before send, and describe a working pre-send tool (hallucination_check.py) whose four verifiers were derived from those cases. The author is one of the people who shipped the slop; the discipline described here exists because the failure happened to him.
The framing throughout is that AI is a calculator: a tool that makes a careful user faster and a careless user wrong faster. The fix is not to disown the calculator; the fix is to apply calculator discipline.
Other
The paper is released under CC BY 4.0. The accompanying tool described in section 6 (hallucination_check.py, approximately 35 KB) is released separately under the BSD 2-Clause Licence and is distributed via the project's public artefacts directory.
Case studies in sections 2 and 3 reference disclosures made to the OpenBSD project (bugs@openbsd.org and security@openbsd.org) during May 2026. Verbatim text of security@openbsd.org correspondence is not reproduced in this paper out of respect for the list's private status; paraphrasing in section 3 preserves the substance.
This paper was drafted with LLM assistance (Claude, Anthropic) as a reasonable adjustment under Equality Act 2010 §20 (neurodivergent author). The author independently verified every cited file path, commit hash, person's name, and URL before publication.
Files
TheCalcDisc.pdf
Files (102.5 kB)
Additional details
- [1] D. Stenberg, "Death by a thousand slops," daniel.haxx.se, 14 July 2025. https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
- [2] D. Stenberg, "The end of the curl bug-bounty," daniel.haxx.se, 26 January 2026. https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/
- [3] D. Stenberg, "AI slop attacks on the curl project," daniel.haxx.se, 18 August 2025. https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-the-curl-project/
- [4] B. Toulas, "Curl ending bug bounty program after flood of AI slop reports," BleepingComputer, 2026. https://www.bleepingcomputer.com/news/security/curl-ending-bug-bounty-program-after-flood-of-ai-slop-reports/
- [5] "AI is drowning software maintainers in junk security reports," Help Net Security, 18 May 2026. https://www.helpnetsecurity.com/2026/05/18/problems-with-ai-assisted-vulnerability-research/
- [6] T. Krazit, "cURL's Daniel Stenberg: AI slop is DDoSing open source," The New Stack, 2026. https://thenewstack.io/curls-daniel-stenberg-ai-is-ddosing-open-source-and-fixing-its-bugs/
- [7] "AI slop got better, so now maintainers have more work," The Register, 6 April 2026. https://www.theregister.com/software/2026/04/06/ai-slop-got-better-so-now-maintainers-have-more-work/5223172