Prompt Fuzzing for Fuzz Driver Generation
PromptFuzz is an automated tool that generates high-quality fuzz drivers for libraries via a fuzz loop constructed on mutating LLMs' prompts. The fuzz loop of PromptFuzz aims to guide the mutation of LLMs' prompts to generate programs that cover more reachable code and explore complex API interrelationships, which are effective for fuzzing.
PromptFuzz is currently regarded as the leading approach for generating fuzz drivers both in academia and industry. The fuzz drivers generated by PromptFuzz achieved a branch coverage of 40.12% on the tested libraries, which is 1.61x greater than OSS-Fuzz and 1.67x greater than Hopper. Besides, PromptFuzz detected 33 valid security bugs from 49 unique crashes.
Relase Notes:
- Release v1.0.0. Support AFLPlusPlus, more libraries and fixed many panic bugs! (2026-04-10)
- Upgrade Clang and LLVM versions to 18! (2025-06-07)
- Upgrade async_openai sdk version to 0.28 to support OPENAI interface new features. (2025-06-05)
- Support OpenAI specification! You can use any LLMs via the standard OPENAI specification. （2025-05-02)
✨Features
- Multiply LLM support: Supports any LLM invocation via the OPENAI Interface Specificatio!.
- Context-based Prompt: Construct LLM prompts with the automatically extracted library context.
- Powerful Sanitization: The program's syntax, semantics, behavior, and coverage are thoroughly analyzed to sanitize the problematic programs.
- Prioritized Mutation: Prioritizes mutating the library API combinations within LLM's prompts to explore complex interrelationships, guided by code coverage.
- Fuzz Driver Exploitation: Infers API constraints using statistics and extends fixed API arguments to receive random bytes from fuzzers.
- Fuzz engine integration: Integrates with grey-box fuzz engine: LibFuzzer.
🏆Trophy
The fuzz drivers generated by PromptFuzz can detect a wide range of bugs, most of which are security bugs. For instances, CVE-2023-6277, CVE-2023-52355 and CVE-2023-52356.
PromptFuzz detects uniquely interesting bugs:
| ID | Library | Buggy Function | Bug Type | Status | Track Link |
|---|---|---|---|---|---|
| 1. | libaom | highbd_8_variance_sse2 | SEGV | Confirmed | 3489 |
| 2. | libaom | av1_rc_update_framerate | Uninitialized Stack | Confirmed | 3509 |
| 3. | libaom | timebase_units_to_ticks | Integer Overflow | Confirmed | 3510 |
| 4. | libaom | encode_without_recode | SEGV | Confirmed | 3534 |
| 5. | libvpx | vp8_peek_si_internal | SEGV | Confirmed | 1817 |
| 6. | libvpx | update_fragments | Buffer Overflow | Confirmed | 1827 |
| 7. | libvpx | vp8e_encode | Integer Overflow | Confirmed | 1828 |
| 8. | libvpx | encode_mb_row | Integer Overflow | Confirmed | 1831 |
| 9. | libvpx | vpx_free_tpl_gop_stats | SEGV | Confirmed | 1837 |
| 10. | libmagic | mkdbname | Buffer Overflow | Confirmed | 481 |
| 11. | libmagic | magic_setparam | Buffer Overflow | Waiting | 482 |
| 12. | libmagic | check_buffer | Buffer Overflow | Confirmed | 483 |
| 13. | libmagic | mget | Integer Overflow | Waiting | 486 |
| 14. | libTIFF | TIFFOpen | OOM | Confirmed | 614 |
| 15. | libTIFF | PixarLogSetupDecode | OOM | Confirmed | 619 |
| 16. | libTIFF | TIFFReadEncodedStrip | OOM | Confirmed | 620 |
| 17. | libTIFF | TIFFReadRGBAImageOriented | OOM | Confirmed | 620 |
| 18. | libTIFF | TIFFRasterScanlineSize64 | OOM | Confirmed | 621 |
| 19. | libTIFF | TIFFReadRGBATileExt | SEGV | Confirmed | 622 |
| 20. | sqlite3 | sqlite3_unlock_notify | Null Pointer crash | Confirmed | e77a5 |
| 21. | sqlite3 | sqlite3_enable_load_extension | Null Pointer crash | Confirmed | 9ce83 |
| 22. | sqlite3 | sqlite3_db_config | Null Pointer crash | Confirmed | 5e3fc |
| 23. | c-ares | config_sortlist | Memory Leak | Confirmed | d62627 |
| 24. | c-ares | config_sortlist | Memory Leak | Confirmed | d62627 |
| 25. | libjpeg-turbo | tj3DecodeYUV8 | Integer Overflow | Confirmed | 78eaf0 |
| 26. | libjpeg-turbo | tj3LoadImage16 | OOM | Confirmed | 735 |
| 27. | libpcap | pcap_create | File Leak | Confirmed | 1233 |
| 28. | libpcap | pcapint_create_interface | Null Pointer crash | Confirmed | 1239 |
| 29. | libpcap | pcapint_fixup_pcap_pkthdr | Misaligned Address | Confirmed | - |
| 30. | cJSON | cJSON_SetNumberHelper | Error Cast | Confirmed | 805 |
| 31. | cJSON | cJSON_CreateNumber | Error Cast | Confirmed | 806 |
| 32. | cJSON | cJSON_DeleteItemFromObjectCaseSensitive | TimeOut | Confirmed | 807 |
| 33. | curl | parseurl | Assertion Failure | Confirmed | 12775 |
Usage
See in Usage
Trouble Shooting
PromptFuzz customized both static and dynamic analysis on the generated harnesses to ensure the harness effectiveness and bug detection accuracy. As LLMs can generate a diverse both C and C++ syntax, our predefined analysis could be panic when it encouters some corner cases. If you encountered that cases, please reproduce that issue with RUST_BACKTRACE=1 and report it. If possible, let your coding pilot (using a powerful LLM) try to identifiy the issues and fix them. Welcome PRs.
🎈Future Works
- Close-source libraries: Apply PromptFuzz to close-source libraries by fine tuning LLMs on private code corpus.
- Generalization: Generalize PromptFuzz to binary programs.