Local Expectation Gradients for Black Box Variational Inference
We introduce local expectation gradients which is a general purpose stochastic variational inference algorithm for constructing stochastic gradients by sampling from the variational distribution. This algorithm divides the problem of estimating the stochastic gradients over multiple variational parameters into smaller sub-tasks so that each sub-task explores intelligently the most relevant part of the variational distribution. This is achieved by performing an exact expectation over the single random variable that most correlates with the variational parameter of interest resulting in a Rao-Blackwellized estimate that has low variance. Our method works efficiently for both continuous and discrete random variables. Furthermore, the proposed algorithm has interesting similarities with Gibbs sampling but at the same time, unlike Gibbs sampling, can be trivially parallelized.
SUMMARY This paper presents an idea that would be very interesting to NIPS readers. It presents a preliminary analysis of an estimator of a quantity that every variational inference researcher would want to know about. The mathematical exposition is good: the authors promise more technical and theoretical studies down the road. The experimental study is the weakest point of the paper, where they miss the opportunity to explore practical aspects of their estimator (specifically, other integration techniques beyond Gaussian quadrature). Overall it is a good paper.QUALITY The paper presents a new method in a careful and well-paced manner. The mathematical development is good. However, the experimental section could use a bit more work. My main comment is that I would have rather had the authors study alternative ways of computing (or approximating) the internal expectation in Equation (9). Instead, they study two uninteresting toy models in Sections 4.1 and 4.2, where it is not clear what insight they expect the reader to extract. The experimental study in Section 4.3 is good, because it addresses a case where ReGrad is not applicable. This is important because there appears to be little reason to use LeGrad for differentiable probability models.CLARITY The paper is well written and accessible.I am not particularly keen on how Section 3.1 is structured. It's not clear to me why the authors have bundled a time-complexity study with the (in my opinion) much more interesting connection to Gibbs sampling. I would rather prefer that they minimize the time-complexity paragraph and push it to the experimental section; then, they could devote this space in the narrative to the connection to Gibbs (and maybe even expand on it some more).Figure 2: caption describes (a) (a) (c). should be (a) (b) (c).Figure 3 also has some caption issues.ORIGINALITY The authors propose a new idea for estimating the gradient of an expectation that pops up in many of the state-of-the-art variational inference techniques proposed in the last few years. This paper should have no trouble on the originality front.SPECIFICS line 151: extra comma in equation, needs reformatting.line 196: ... that IT has with ...line 250: ... we consider --a-- sigmoid belief networks ...