Towards Robust Benchmarking of Quantum Optimization Algorithms
<|reference_start|>Towards Robust Benchmarking of Quantum Optimization Algorithms: Benchmarking the performance of quantum optimization algorithms is crucial for identifying utility for industry-relevant use cases. Benchmarking processes vary between optimization applications and depend on user-specified goals. The heuristic nature of quantum algorithms poses challenges, especially when comparing to classical counterparts. A key problem in existing benchmarking frameworks is the lack of equal effort in optimizing for the best quantum and, respectively, classical approaches. This paper presents a comprehensive set of guidelines comprising universal steps towards fair benchmarks. We discuss (1) application-specific algorithm choice, ensuring every solver is provided with the most fitting mathematical formulation of a problem; (2) the selection of benchmark data, including hard instances and real-world samples; (3) the choice of a suitable holistic figure of merit, like time-to-solution or solution quality within time constraints; and (4) equitable hyperparameter training to eliminate bias towards a particular method. The proposed guidelines are tested across three benchmarking scenarios, utilizing the Max-Cut (MC) and Travelling Salesperson Problem (TSP). The benchmarks employ classical mathematical algorithms, such as Branch-and-Cut (BNC) solvers, classical heuristics, Quantum Annealing (QA), and the Quantum Approximate Optimization Algorithm (QAOA).<|reference_end|>
@article{bucher2024towards,
    title={Towards Robust Benchmarking of Quantum Optimization Algorithms},
    author={David Bucher, Nico Kraus, Jonas Blenninger, Michael Lachner, Jonas
  Stein and Claudia Linnhoff-Popien},
    journal={arXiv preprint arXiv:2405.07624},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.07624},
    primaryClass={quant-ph cs.ET}
}