Model evidence from nonequilibrium simulations
The marginal likelihood, or model evidence, is a key quantity in Bayesian parameter estimation and model comparison. For many probabilistic models, computation of the marginal likelihood is challenging, because it involves a sum or integral over an enormous parameter space. Markov chain Monte Carlo (MCMC) is a powerful approach to compute marginal likelihoods. Various MCMC algorithms and evidence estimators have been proposed in the literature. Here we discuss the use of nonequilibrium techniques for estimating the marginal likelihood. Nonequilibrium estimators build on recent developments in statistical physics and are known as annealed importance sampling (AIS) and reverse AIS in probabilistic machine learning. We introduce estimators for the model evidence that combine forward and backward simulations and show for various challenging models that the evidence estimators outperform forward and reverse AIS.
This paper discusses tackling marginal likelihood
estimation, a central problem in Bayesian model selection, using
insights provided by so-called nonequilibrium fluctuation
theorems. While Jarzynksi's Equality (JE) allows a natural sequential
Monte Carlo estimator of log Z, the estimator can be high-variance if
the resultant sequence of nonequilibrium distributions have
insufficient overlap, which is a common problem for complex,
multimodal problems. The fluctuation theorem allows (under generally
non-realistic assumptions; more shortly) a bounding of the estimate no
matter how fast the annealing schedule is. In addition to providing
bounds, the reverse work distribution can be incorporated into
estimators for log Z itself that are (hopefully) an improvement over
the one provided by JE. Experiments are run on various statistical
models.
I found this to be an interesting paper and enjoyed reading it. My
main reservations are concerned with the feasibility of simulating
the reverse process in most situations that are
practically-challenging. It appears that the authors do recognize this
limitation, but provide a brief and not overly satisfying discussion
at the end of p.4. Most of the experimental results focus on problems
with a simple low-energy structure (the ferromagnetic Ising and Potts
models, and mixture models with 3 components) The RBM example is a bit
more intriguing, but when trained on MNIST, one can reasonably infer
the existence of a relatively small number of modes. In general, the
method assumes access to an oracle providing a perfectly-trained
model, where the data points do indeed represent the modes of the
distribution (1000 sweeps of Gibbs sampling at low temperature will
move only locally and cannot be expected to move the state too far
from the training points.) While the agreement of the current method
and PT log Z results in this case is encouraging, the fit of the model
is generally difficult to verify in practice- indeed that's the
reason for conducting the simulation in the first place.
To phrase the objections in another way, optimization is generally a
very different problem from sampling, as represented by the
distinction between problems in NP and those in #P. There exist
several interesting types of problems for which computing the ground
state energy is tractable, but counting the number of such minima (obtaining the
ground state entropy) is not. Generating samples uniformly from the
ground state distribution, as would be required for the reverse
methodologies, will then be problematic.
Another issue was that on first reading, it was not immediately clear
what the paper's contributions are; presumably, it is the combined
forward/reverse weighted estimator presented on page 5? The remaining
ones have been previously introduced in the literature. While valid,
I'm ambivalent as to whether this contribution warrants an entire NIPS
paper. Certainly, the connection between marginal likelihood and free
energy estimation is widely-known among Bayesian computational
statisticians.
Despite these misgivings, I still found the paper to be quite cool and
am encouraged to see progress in this research area.
Technical correctness:
-------------------
The paper appears to be technically correct to my knowledge.
Clarity:
-------
The paper is quite well-written and easy to follow for someone
familiar with Sequential Monte Carlo and Bayesian statistics. The
presentation could be better-adapted to the NIPS context, however, as
physics-flavored notions such as "work" may seem distant to a machine
learning audience.
Specific issues:
---------------
252-253: In what sense is simulation of marginal RBM 3x faster than
that of the joint? The joint simulation can be parallelized and
simulated on a GPU due to the conditional independence structure of
the RBM. The statement should be clarified.