VAE Learning via Stein Variational Gradient Descent
A new method for learning variational autoencoders (VAEs) is developed, based on Stein variational gradient descent. A key advantage of this approach is that one need not make parametric assumptions about the form of the encoder distribution. Performance is further enhanced by integrating the proposed encoder with importance sampling. Excellent performance is demonstrated across multiple unsupervised and semi-supervised problems, including semi-supervised analysis of the ImageNet data, demonstrating the scalability of the model to large datasets.
%%% UPDATE: Thank you for your response, which has been read %%%
%%% The score I gave has increased by one level, after I was assured that this work is of some interest to research on VAE %%%
%%% Please ensure that ALL details needed to reproduce the experiments are included in any revision of the paper %%%
This paper contains a lot of different ideas around variational auto-encoders and Stein's method, which are brought together in a way that appears novel. However I feel that the presentation could be improved and more details about the (complicated) implementation should have been included.
- The background on variational auto-encoders was not clear, to this reviewer who has not worked with VAEs. For instance, first p(x | z_n ; \theta) is called the "encoder", then q(z | x ; \phi) is called the "encoder" - which is it, and why is it given this name? 
- \top should be used as transpose in latex.
- What is the domain of \theta? Related, how is \grad_\theta \psi(\theta ; D) defined if both \theta and \psi are vectors?
- Several approximations are composed en route to the final method that is proposed. It is unclear how all these approximations interact - this isn't assessed theoretically - so that all of the assessment must be based on the empirical results. 
- Unfortunately several details necessary to reproduce the results seem to have been omitted. e.g. the value of \epsilon (step size for SVGD), the number of SVGD iterations (or the stopping criterion), the initial locations of the particles, etc. In particular, the initial location of the particles would presumably be quite influential to the performance of SVGD.
- I did not understand Theorem 2. The claim seems to be that KL_{q,p}^k is involved in a tighter variational lower bound, but what is the significance of the fact that Lim_{k \rightarrow \infty} KL_{q,p}^k(\Theta ; D) = 0? This is not really explained.
- On line 178, "q_0" has not been defined, I think.
- In general there are a lot of moving parts in the proposed method, and most of these "levers" are unexplored in the experiments. For instance, how does the choice of k impact on performance?
- An MCMC benchmark is used for a couple of the experiments - there is presumably an implicit argument that the proposed method is faster than MCMC. However, this isn't actually tested and computational times against MCMC are not included.