Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling
Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning. The Markov Chain Monte Carlo procedures that are used are often discrete-time analogues of associated stochastic differential equations (SDEs). These SDEs are guaranteed to leave invariant the required posterior distribution. An area of current research addresses the computational benefits of stochastic gradient methods in this setting. Existing techniques rely on estimating the variance or covariance of the subsampling error, and typically assume constant variance. In this article, we propose a covariance-controlled adaptive Langevin thermostat that can effectively dissipate parameter-dependent noise while maintaining a desired target distribution. The proposed method achieves a substantial speedup over popular alternative schemes for large-scale machine learning applications.
Paper Title: Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian SamplingPaper Summary: This paper presents a new method (the "covariance-controlled adaptive Langevin thermostat") for MCMC posterior sampling for Bayesian inference. Along the lines of previous work in scalable MCMC, this is a stochastic gradient sampling method. The presented method aims to decrease parameter-dependent noise (in order to speed-up convergence to the given invariant distribution of the Markov chain, and generate beneficial samples more efficiently), while maintaining the desired invariant distribution of the Markov chain. Similar to existing stochastic gradient MCMC methods, this method aims to find use in large-scale machine learning settings (i.e. Bayesian inference with large numbers of observations). Experiments on three models (a normal-gamma model, Bayesian logistic regression, and a discriminative restricted Boltzmann machine) aim to show that the presented method performs better than stochastic gradient Hamiltonian monte carlo (SGHMC) and stochastic gradient Nose-Hoover thermostat (SGNHT), two similar existing methods. Comments:- I feel that this paper proposes a valid contribution to the area of stochastic gradient MCMC methods, and does a good job putting this method in context with similar previous methods (SGHMC and SGNHT). However, one detriment of this paper is that it is somewhat incremental, both in terms of ideas and the results shown.- In the experiments, comparisons are only against two methods: SGHMC and SGNHT. It might be nice to also see results for some of the other recently developed mini-batch MCMC methods, (such as the original stochastic gradient Langevin dynamics, or stochastic gradient Fisher scoring), or for some of the methods that do not rely on stochastic gradients, such as: (Bardenet, Remi, Arnaud Doucet, and Chris Holmes. "On Markov chain Monte Carlo methods for tall data." arXiv preprint arXiv:1505.02827 (2015)), or (Maclaurin, Dougal, and Ryan P. Adams. "Firefly Monte Carlo: Exact MCMC with subsets of data." arXiv preprint arXiv:1403.5693 (2014)).- In Figure 1, the inset "peaks" add very little to the figure -- they seem to be an only very-slight zoom into what is shown in the non-inset part of the figure.- I feel there are a few places in this paper where the quality of writing could be improved. In the abstract, there are few sentences that feel somewhat ambiguous to me (such as "one area of current research asks how to utilise the computational benefits of stochastic gradient methods in this setting."). In the intro (second paragraph), the order of presentation of stochastic gradient methods seems odd (first, the collection of all existing methods are described, and then afterwards, the first developed method is described). In section 2, there is a bit of confusion when a few terms are introduced without enough description (such as "temperature", "Boltzmann constant"); it would be better to give a brief description or intuition when introducing these terms to a machine learning audience.