Bayesian Sampling Using Stochastic Gradient Thermostats
Dynamics-based sampling methods, such as Hybrid Monte Carlo (HMC) and Langevin dynamics (LD), are commonly used to sample target distributions. Recently, such approaches have been combined with stochastic gradient techniques to increase sampling efficiency when dealing with large datasets. An outstanding problem with this approach is that the stochastic gradient introduces an unknown amount of noise which can prevent proper sampling after discretization. To remedy this problem, we show that one can leverage a small number of additional variables to stabilize momentum fluctuations induced by the unknown noise. Our method is inspired by the idea of a thermostat in statistical physics and is justified by a general theory.
The Firefly paper is interesting, but to me it looked like a bunch ofwork for a tiny speed-up: I wouldn't recommend practitioners bother. Itisn't clear that developments in that direction will be useful either,so it's not time to shut down brainstorming of other ideas.Welling and Teh's work has had impact. I don't think it's particularlyjustified except in the limit of small step sizes either? Thejustification of past work has largely been empirical. I think thispaper's three varied and moderate-sized applications is good enough.The benefits aren't huge, but they are there.Negatives:While I can infer what is meant in some of the places thatAssigned_Reviewer_31 queries, I agree that the paper is hard to follow.The core idea is barely explained at all. The background section and thematerial around equation (4) would benefit from revision.Given the exposition in the paper, it's perhaps surprising that the 1Dexample (n=1) works out so well, given that above (4) it says that n islarge. However, in this example, things are favourable because having ascalar \xi is ok. I'm more worried about the high-dimensional case, whensampling noise can upset the momentum in different directionsdifferently. Then fixing up the scalar p'*p/n isn't enough to samplefrom the correct distribution. (Acknowledged around the end of Section4.2, but could be made clearer / more prominent in discussion.)Final opinion:It's an interesting approach, with someweaknesses in the paper. The sampler won't be correct, but theexperimental evaluation shows incremental improvement over past work.The dynamics: The presentation of SDE's is hard going, and could beimproved with a discussion of the stability of the discrete-timealgorithm. I don't think their potentially-unstable choice of discretedynamics is necessarily fatal if it works (although a discussion wouldbe nice). I'm personally more concerned theoretically with the posteriorbeing wrong due to non-isotropic noise.