Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.
The authors derive a new inference algorithm for bayesian inference connecting variational inference to recent results  using Kernelized Stein discrepancy. The algorithm is particle based and smoothly interpolates between MAP and true Bayesian posterior (by using more particles).  At a high level, the authors connect the derivative of a small perturbation (x+eps phi(x), which are going to be progressively used to transform an initial distribution into the target distribution) to the stein operator. Then, by restricting the deformation from identity to be a function in a RKHS, they can obtain a closed-form for the optimal deformation function phi, which takes the form of a stein operator on the initial distribution. Distributions are represented by samples and updated according to the simple operator. 
Overall, I found the paper interesting; the paper offers new theory as well as numerical results comparable to the state of the art on decently difficult datasets.  Perhaps due to space constraints, an important part of the paper (section 3.2) - the inference algorithm - is poorly explained. In particular, I initially thought that the use of particles meant that the approximating distribution was a sum of Dirac delta functions - but that cannot be the case since, even with many particles, the 'posterior' would degenerate into the MAP (note that in similar work, authors either use particles when p(x) involves discrete x variables, as in Kulkarni et al,  or 'smooth' the particles to approximate a continuous distribution, as in Gershman et al). Instead, it looks like the algorithm works directly on samples of the distribution q0, q1.. (hence the vague 'for whatever distribution q that {xi}ni=1 currently represents'). It is tempting to consider q_i to be a kernel density estimate (mixture of normals with fixed width), and see if we can approximate equation 9 for that representation to be stable. Is this possible / something the authors have considered?  Also, since the algorithm does not care about what precise form the distribution q takes, but the algorithm cannot possibly work for q sum of delta functions, I assume there are some smoothness assumptions required for the theorem to hold (even if it is simple as 'q has a density').   Minor: - In the second paragraph of the introduction, the authors explain that in some work (Kingma and Welling, Rezende and Mohamed, etc.), neural networks are used to approximate a posterior distribution; this is not accurate: in those papers the posterior is typically a simple distribution (often, gaussian mean-field);  the neural network is used to amortize inference and predict the parameters of the posterior as  a function of the data. This is faster, but theoretically worse than regular mean-field inference (optimization issues put aside). Rezende and Mohamed 2015 is, however, truly using a neural network to parametrize a more complex posterior distribution.  - The connection to the reparametrization trick is very promising and would have deserved a few words (even if future work goes into mode details), at least in the appendix, where the reparametrization is key in obtaining 3.1.