Meta-Learning MCMC Proposals
Effective implementations of sampling-based probabilistic inference often require manually constructed, model-specific proposals. Inspired by recent progresses in meta-learning for training learning agents that can generalize to unseen environments, we propose a meta-learning approach to building effective and generalizable MCMC proposals. We parametrize the proposal as a neural network to provide fast approximations to block Gibbs conditionals. The learned neural proposals generalize to occurrences of common structural motifs across different models, allowing for the construction of a library of learned inference primitives that can accelerate inference on unseen models with no model-specific training required. We explore several applications including open-universe Gaussian mixture models, in which our learned proposals outperform a hand-tuned sampler, and a real-world named entity recognition task, in which our sampler yields higher final F1 scores than classical single-site Gibbs sampling.
This paper describes a method to learn smart proposals for speeding up MCMC inference in probabilistic graphical models (PGMs). The main idea is to decompose the graph into possibly overlapping "structural motifs" such as grids or chains with similar relationships between the variables that are either proposed variables, conditioned variables, or local parameters. Using an appropriate distribution, samples are generated for training a probability distribution on the proposed variables given the conditioned variables and local parameters for each instantiation of the motif in the PGM. These learned proposal distributions are then used to generate proposals which are then accepted/rejected using the MH rule. (Variables not covered by any motif are sampled using single-site Gibbs).  This work is very similar to inference compilation cited as [19] in the paper. However, the big difference is that inference compilation works on training a proposer for the entire probabilistic program whereas this paper builds up the proposer from multiple parts where each part is trained on a smaller sub-structure in the probabilistic model. The hope being that training a proposer on smaller sub-structures would allow those to be reused across models.  Overall the work is quite sound and the results are very promising. The main reason for my low overall rating is to do with the disparity between the claims made in the abstract/introduction with the actual contents of the text. The authors claim, for example, that the "The learned neural proposals generalize to occurrences of common structural motifs across different models, allowing for the construction of a library of learned inference primitives that can accelerate inference on unseen models with no model-specific training required." However, there is no evidence given in the text of a proposer being trained on one model and then used for inference on another one.  The other drawback of the paper is that the authors seem to suggest that they have a magical blackbox method that works in all models without any hand tuning as opposed to other techniques (line 51). But in every experiment described their appears to be a fair amount of hand tuning in terms of figuring out how to generate the data for training the proposers. In one case they decide to collapse out a large number of the variables all together! This is more of a collection of recipes rather than a systematic procedure for training the proposers.   Finally, it appears that the work is not applicable beyond the realm of fixed size models for two reasons. 1) Even though the authors mention probabilistic programming languages (PPLs) early in the introduction there is no follow up in the text. For example, one would expect some discussion about extracting motifs in the context of a concrete PPL. Also, the paper assumes that the motifs can be extracted at the beginning of inference (Algorithm 1). However, in PPLs the graph structure changes with different values of the latent variables, so it doesn't appear applicable. 2) I don't buy the open universe probabilistic model (OUPM) example in the text. Instead of sampling the number of objects in the universe either directly or lazily (through a Dirichlet process) this example has an upper limit on the number of objects and assigns an "activity" indicator to each possible object with the total number of objects defined as the sum of the active objects. This is an oversimplification. OUPMs are meant to capture not just an unknown number of objects, but also potentially unbounded number of objects. There is nothing in the text of this paper that describes how a proposer would intelligently determine adding or subtracting objects from the model.  Despite these drawbacks I believe that the work describes a significant improvement over single-site Gibbs for fixed size models with discrete-valued variables, and is worth publishing.   More detailed comments:  - The toy example in Fig 1 is misleading. It seems to suggest that the Neural Network is trained only on the local model parameters alpha and beta. While in reality it appears that the neural network is trained on the local model parameters as well as the conditioning variables.  - It is truly baffling that the conditioning variables for a motif might not include the Markov Blanket of the proposed variables. This appears to run contrary to the theme of the paper that the proposers learned from these motifs are reusable. For example this work is attempting to approximate the probability distribution p_\psi_i(B_i|C_i) at the beginning of inference, but this distribution depends on the values of the remaining variables in MB(B_i) - C_i which can change as inference progresses. In other words, it appears that the proposal that is learned may not even be relevant from sample to sample let alone be reused across models. The GMM is the only example in the experiments where the Markov Blanket is not a subset of the conditioning set. In the GMM case I suspect that the n=60 randomly selected samples that are part of the motif are a good enough approximation of the Markov Blanket. It would have been helpful to demonstrate more clearly with an example why the authors decided that C_i doesn't equal MB(B_i).  - Line 91 talks about "training with random model parameterization", but this is not a well developed concept in this paper. There are a series of examples in the experiments, but no crisp procedure.  - Line 208, 209. It is a stretch to conclude from one observed low KL-divergence in this small grid example that this approach works in general to approximate true conditionals. Also, it is not clear from the text if the inference used a single proposal for all occurrences of the grid motif in the graph or were separate proposers trained for each grid motif. The other unanswered question is the degree of overlap between grids, and whether that affects inference.  Equation (4) is a bit confusing. This is describing a distribution for a binary valued variable, right? So why do we have "[0,1]" on the first line and "[1,0]" on the second line instead of say "0" on the first line and "1" on the second line?  - Line 252. It is not entirely clear how the samples for training the proposer were obtained. Is the model being simulated to generate samples for the training data (as in [19])?   Minor:  Line 162. I am assuming that a probability table for a bounded discrete table is simply a categorical distribution over the possible values. So, then why do we need a mixture of these. Can't a mixture of categoricals be represented as a single categorical?  1 (a) -> om -> on  Post-Response:  I am willing to accept the authors response that collapsing out the GMM is not necessary to get better performance, and that they will provide results without collapsing in the next version. However, the claim that OUPMs can be handled with RNNs requires more scrutiny, and such claims should not be added without review. For the current NIPS'18 submission, the discussion on OUPMs and PPLs properly belongs in the "future work" section of the paper given the sketchy details.  The paper still has a number of important under-developed concepts such as not using the Markov Blanket for the conditioning set. The authors haven't explained why this choice of the conditioning set doesn't affect reusability of the trained proposers.