  Production systems use heuristics because they are faster or scale better
than their optimal counterparts. Yet, practitioners are often unaware of the
performance gap between a heuristic and the optimum or between two heuristics
in realistic scenarios. We present MetaOpt, a system that helps analyze
heuristics. Users specify the heuristic and the optimal (or another heuristic)
as input, and MetaOpt automatically encodes these efficiently for a solver to
find performance gaps and their corresponding adversarial inputs. Its suite of
built-in optimizations helps it scale its analysis to practical problem sizes.
To show it is versatile, we used MetaOpt to analyze heuristics from three
domains (traffic engineering, vector bin packing, and packet scheduling). We
found a production traffic engineering heuristic can require 30% more capacity
than the optimal to satisfy realistic demands. Based on the patterns in the
adversarial inputs MetaOpt produced, we modified the heuristic to reduce its
performance gap by 12.5$\times$. We examined adversarial inputs to a vector bin
packing heuristic and proved a new lower bound on its performance.

Networking and Internet Architecture;Computer Science and Game Theory