abstract
approximate probabilistic inference algorithms are central to many fields. examples include sequential monte carlo inference in robotics, variational inference
in machine learning, and markov chain monte carlo inference in statistics. a
key problem faced by practitioners is measuring the accuracy of an approximate
inference algorithm on a specific data set. this paper introduces the auxiliary
inference divergence estimator (aide), an algorithm for measuring the accuracy of
approximate inference algorithms. aide is based on the observation that inference
algorithms can be treated as probabilistic models and the random variables used
within the inference algorithm can be viewed as auxiliary variables. this view leads
to a new estimator for the symmetric kl divergence between the approximating
distributions of two inference algorithms. the paper illustrates application of aide
to algorithms for inference in regression, hidden markov, and dirichlet process
mixture models. the experiments show that aide captures the qualitative behavior
of a broad class of inference algorithms and can detect failure modes of inference
algorithms that are missed by standard heuristics.