Paper 2026/1107

KAT-Seeded Fuzzing of Stateful Hash-Based Signature Verification in liboqs

Muhammad Ibrahim, Quentangle

Muhammed Sihan Haroon, Quentangle

Abstract

Stateful hash-based signature schemes such as XMSS and LMS are increasingly important in post-quantum cryptographic deployments, yet their verification paths remain difficult to fuzz effectively because key generation is substantially more expensive than verification. This paper presents a structured libFuzzer methodology for testing stateful hash-based signature verification in liboqs. The proposed approach uses pre-computed Known Answer Test (KAT) vectors to initialise valid public key, signature, and message buffers once during harness setup, enabling high-throughput mutation of verification inputs without repeated key generation. Using this methodology, we implemented and upstreamed two fuzz harnesses targeting XMSS/XMSSMT and LMS/HSS-LMS verification paths in liboqs. A ten-minute AddressSanitizer-enabled fuzzing campaign identified an OID-confusion heap-buffer-overflow in xmssmt_core_sign_open, assigned CVE-2026-46344. The flaw occurs when attacker-controlled public-key OID bytes derive a parameter set whose expected signature length exceeds the caller-allocated buffer, resulting in an out-of-bounds read during signature parsing. The issue was reproduced using a five-byte minimal crashing input and a standalone C reproducer, disclosed through the Open Quantum Safe coordinated disclosure process, and fixed in liboqs 0.16.0. We further identify a mutation-distribution bias affecting short-input LMS fuzzing and resolve it using a round-robin field-rotation strategy. The resulting framework provides a reproducible and upstream-integrated methodology for adversarial testing of stateful post-quantum signature verification implementations.

@misc{cryptoeprint:2026/1107,
      author = {Vishnu Ajith and Muhammad Ibrahim and Muhammed Sihan Haroon},
      title = {{KAT}-Seeded Fuzzing of Stateful Hash-Based Signature Verification in liboqs},
      howpublished = {Cryptology {ePrint} Archive, Paper 2026/1107},
      year = {2026},
      url = {https://eprint.iacr.org/2026/1107}
}