TL;DR: The mutation-generated faults and curated benchmarks long used to evaluate quantum software testing tools may be missing 86.3% of the bugs that actually show up in real quantum projects.
Title: Are We Testing the Wrong Quantum Bugs? A Study of Fault Representation Gaps in Quantum Software Testing
URL:
Key points
🔍 Collected 410 verified real-world bugs from 10 repositories (Qiskit, PennyLane, Cirq, and more), classified into 56 bug families
🧬 Mutation-testing tools only cover 20 of those families
📚 Curated benchmarks add just 15 more families
📉 Only 13.7% of real bugs fall into families represented by either source
🧩 The big missing categories: compiler transformation errors (66 bugs), type-handling errors (53), missing input validation (47)
⚙️ Most of these faults live in compilers, intermediate representations, and framework interactions, not in the circuits themselves, so circuit mutation alone can't reproduce them
The reminder that a correct circuit says nothing about a correct compilation feels like a lesson that reaches well beyond quantum software testing.
#
QuantumComputing# #
SoftwareTesting#