A Stein variational Newton method
Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reproducing kernel Hilbert space. In this paper, we accelerate and generalize the SVGD algorithm by including second-order information, thereby approximating a Newton-like iteration in function space. We also show how second-order information can lead to more effective choices of kernel. We observe significant computational gains over the original SVGD algorithm in multiple test cases.
The authors propose an version of Stein variational gradient descent (SVGD) that uses approximate second-order information, both to improve the optimization (by analogy with other quasi-Newton optimization methods) and to create a geometry-aware kernel (by analogy with pre-conditioning).  The use of second order information improves convergence considerably over standard SVGD, with the greatest benefit provided by both the geometry-aware kernels and second-order optimization steps.  Stein methods for approximate inference are an exciting topic these days, the improvements are both practically feasible and significant, and the paper is very well-written.  I will argue that the paper should be accepted and really only have minor comments and suggestions. - A more appropriate title is “A Stein variational Quasi-Newton method”, since the authors are not actually able to take a full Newton step, even on small problems, due to the fact that the transforms, and hence push-forward density, are not available in closed form (line 128). - In general, I might replace the phrase “inexact Newton” with “quasi-Newton” for continuity with other optimiziation literature (e.g. line 130). - “There exist infinitely many such maps” (line 62) is repeated from the introduction. - At line 80, you can define equation (6) without Frechet differentiability; it is equation (7) that requires it (specifically, that \nabla J_{p_l} does not depend on V).  So for clarity perhaps it would be better to move the assumption of Frechet differentiability to line 81. - I think it might be better to spell out the definition of N(x) at line 131.  I gathered that it is the outer product of the gradient of log p(x) at x, but it would be better to say that explicitly since this is an important part of the algorithm.  Calling it equivalent to the Fisher information matrix is confusing -- the Fisher information matrix is usually as the covariance of the scores, whereas I think the authors don’t intend any expectation to be calculated.  Furthermore, a correctly specified Fisher information matrix could be calculated either from the scores or from the Hessian, and the authors want to emphasize that it is the former, not the latter. - At line 150, the authors define a matrix, not a metric, so it would be better to call it that.  This matrix is later used to /form/ a metric. - At line 236, is it supposed to be F(x), not F(u)?  In line 238 F is applied to the Brownian motion, not the solution to the SDE. - Line 254, for comparison I’d be interested to know how long HMC took.