Automatic Variational Inference in Stan
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult for non-experts to use. We propose an automatic variational inference algorithm, automatic differentiation variational inference ( ); we implement it in Stan (code available), a probabilistic programming system. In the user provides a Bayesian model and a dataset, nothing else. We make no conjugacy assumptions and support a broad class of models. The algorithm automatically determines an appropriate variational family and optimizes the variational objective. We compare to sampling across hierarchical generalized linear models, nonconjugate matrix factorization, and a mixture model. We train the mixture model on a quarter million images. With we can use variational inference on any model we write in Stan.
Review Summary ============== Score: 8 top 50% of NIPS papersI'm excited about the core ideas in this paper: it's a huge step towards making variational inference general-purpose at scale for a big class of models. I think it belongs at NIPS, especially if the authors can address some concerns about local optima and revise the paper to include a new plot with a more intuitive illustrative example.Summary of Paper ================ This paper develops a method for automatic variational inference in the Stan probabilistic programming language, which can support any model that has a differentiable log joint probability. The proposed method works by transforming all latent variables to unconstrained real values, and then approximating their posterior with a Gaussian with diagonal covariance (the diagonal assumption corresponds to an mean-field independence assumption on every scalar in the parameter vector). A further transformation standardizes each 1D Gaussian to a standard normal (zero mean, unit variance), so that Monte Carlo integration can be used to estimate the expectations involved in the variational objective function. To scale to massive datasets, the authors employ stochastic variational inference with an adaptive learning rate. Experiments show that the method provides better predictions on heldout data than Stan's MCMC algorithm for a variety of nonconjugate models (regression, matrix factorization, and mixtures). Reviewer Comments on Method =========================== Overall, I like the general approach used. I have a few key issues I'd like addressed in the rebuttal, in order of importance.Issue M1: I'd like to see at least some discussion of how vulnerable this method is to local optima. My guess is, that like other variational methods, the answer is extremely sensitive. It would be interesting to hear if the authors have any strategies to avoid this sensitivity, aside from multiple random initializations.Issue M2: I'm concerned that the monte-carlo integration strategy may be very noisy and lead to slow convergence. Previous work on black-box variational (Ranganath, Gerish and Blei 2014) indicated that without some strategies to reduce variance (rao-blackwell-ization and control variate methods), simple monte carlo estimates to obtain gradients were quite terrible -- see esp. their Fig. 2. I'd like to know if this kind of diagnostic plot, or any other, was ever made. The experiments seem to converge OK, but I find it really surprising that a single sample (M=1), as used in all experiments, is sufficient.Issue M3: I'd like to see more discussion of how the method is initialized. This is a crucial for variational inference methods that get stuck easily in local optima. Have different strategies been tried? Issue M4: How to select batch size? It seems like the growing consensus is that very small batches will lead to poor models, which makes sense. But is there any advice to practioners about how to select an appropriate batch size? Perhaps just use the largest size you can afford, doing full-dataset inference if feasible? Reviewer Comments on Experiments ================================ Issue E1: Internal experiments should show many (at least 5, preferablly more like 20 or 50) runs of each method (each from an independent initialization), rather than just one run. Local optima are very prevalent in inference, and plots of multiple runs can help diagnose how sensitive each method (MCMC/ADVI) is.Issue E2: Experiments are missing one "intuition-building" plot, that shows the estimated parameters in a case for which the good or ideal parameters are "obvious" from inspection of the raw data. I like the faces in Fig 5c, but its hard to tell if this is a "good" result. Instead, I recommend something like a classic mixture model on 2D scatterplot data with a few obvious clusters, some of which have non-trivial covariance. The result from ADVI should be some Gaussian blobs centered on cluster means but with diagonal covariances (due to mean-field), while the MCMC result maybe be more accurate (no independence assumptions) but probably slower. I think this will help readers understand the tradeoffs of mean-field, etc. It's often really hard to understand from traceplots whether a given run is "good" in an absolute sense.Issue E3: In Line 414, "label switching" is blamed for the poor performance of MCMC for the mixture model. I don't think I buy this explanation. The fact that cluster indices can be permutted over many iterations doesn't explain poor predictive performance, because prediction scores should be irrelevant to the order of the clusters. Instead, the problem is probably that the MCMC sampler is stuck in a local optima and cannot escape. This is the kind of thing that is best diagnosed with the plot from E2. Less Critical Issues (need not reply to in rebuttal) ---------------------------------------------------- Issue M5: Variational and MCMC are often seen as either/or choices, but I'm often interested in whether variational could be used to rapidly get to a decent estimate, and then MCMC is used to get a more refined, reliable estimate. Is it possible to initialize an MCMC chain in Stan with the output of variational? Or vice versa?Issue E4: This may be too much to ask, but I think it would be nice to include one comparison to an external method (variational probably, but MCMC could work too) that has been custom-designed for some specific data and model. For many readers, the question of whether they should use Stan or roll-their-own custom solution may be relevant, and it would be interesting to show a case for which the Stan approach reaches comparable performance without the cost of custom inference. Significance ============ This project has great potential for wide-adoption, given its complete integration into the increasingly-popular Stan language. Novelty ======= Developing the proposed general-purpose framework and integrating it into Stan is definitely a very new and worthwhile contribution.  Clarity ======= The paper has a crisp, simple style that I found very accessible. Aside from a few unclear sentences (detailed below), I think the overall organization and style do not need heavy revision. Line-by-line Comments ===================== (not necessary to respond to these in rebuttal)Line 257: I found the phrase "we cannot directly represent it as a computer program" a bit confusing. Perhaps just leave it out. The sentence about an intractable integral is enough.Supplement E, Line 589: You give ref 3 (the SVI JMLR article) as a reference for adagrad, but in fact that paper has very little material on adaptive gradients, aside from a reference other papers. Please fix.