Published as a conference paper at ICLR 2018 GRADIENT ESTIMATORS FOR IMPLICIT MODELS
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions. The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models. This paper alleviates the need for such approximations by proposing the Stein gradient estimator, which directly estimates the score function of the implicitly defined distribution. The efficacy of the proposed estimator is empirically demonstrated by examples that include gradient-free MCMC, meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.
Post rebuttal phase (see below for original comments)
================================================================================
I thank the authors for revising the manuscript. The methods makes sense now, and I think its quite interesting. While I do have some concerns (e.g. choice of eta, batching may not produce a consistent gradient estimator etc.), I  think the paper should be accepted. I have revised my score accordingly.
That said, the presentation (esp in Section 2) needs to be improved. The main problem is that many symbols have been used without being defined. e.g. phi, q_phi, \pi,  and a few more. While the authors might assume that this is obvious, it can be tricky to a reader - esp. someone like me who is not familiar with GANs. In addition, the derivation of the estimator in Section 3 was also sloppy. There are neater ways to derive this using RKHS theory without doing this on a d' dimensional space.
Revised summary: The authors present a method for estimating the gradient of some training objective for generative models used to sample data, such as GANs. The idea is that this can be used in a training procedure. The idea is based off the Stein's identity, for which the authors propose a kernelized solution. The key insight comes from rewriting the variational lower bound so that we are left with having to compute the gradients w.r.t a random variable and then applying Stein's identity. The authors present applications in Bayesian NNs and GANs.
Summary
================================================================
The authors present a method for estimating the gradient of some training objective
for generative models used to sample data, such as GANs. The idea is that this can be
used in a training procedure. The idea is based off the Stein's identity, for which the
authors propose a kernelized solution. The authors present applications in Bayesian NNs
and GANs.

Detailed Reviews
================================================================
My main concern is what I raised via a comment, for which I have not received a response
as yet. It seems that you want the gradients w.r.t the parameters phi in (3). But the
line immediately after claims that you need the gradients w.r.t the domain of a random
variable z and the subsequent sections focus on the gradients of the log density with
respect to the domain. I am not quite following the connection here.
Also, it doesn't help that many of the symbols on page 2 which elucidates the set up
have not been defined. What are the quantities phi, q, q_phi, epsilon, and pi?
Presentation
- Bottom row in Figure 1 needs to be labeled. I eventually figured that the colors
  correspond to the figures above, but a reader is easily confused.
- As someone who is not familiar with BNNs, I found the description in Section 4.2
  inadequate.
 
Some practical concerns:
- The fact that we need to construct a kernel matrix is concerning. Have you tried
  batch verstions of these estimator which update the gradients with a few data points?
- How is the parameter \eta chosen in practice? Can you comment on the values that you
  used and how it compared to the eigenvalues of the kernel matrix?
Minor
- What is the purpose behind sections 3.1 and 3.2? They don't seem pertinent to the rest
  of the exposition. Same goes for section 3.5? I don't see the authors using the
  gradient estimators for out-of-sample points?
I am giving an indifferent score mostly because I did not follow most of the details.