GO GRADIENT FOR EXPECTATION-BASED OBJECTIVES
Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters γ for expectation-based objectives E qγ (y) [f (y)]. Most existing methods either (i) suffer from high variance, seeking help from (often) complicated variance-reduction techniques; or (ii) they only apply to reparameterizable continuous random variables and employ a reparameterization trick. To address these limitations, we propose a General and One-sample (GO) gradient that (i) applies to many distributions associated with non-reparameterizable continuous or discrete random variables, and (ii) has the same low-variance as the reparameterization trick. We find that the GO gradient often works well in practice based on only one Monte Carlo sample (although one can of course use more samples if desired). Alongside the GO gradient, we develop a means of propagating the chain rule through distributions, yielding statistical back-propagation, coupling neural networks to common random variables.
* Summary
The paper proposes an improved method for computing derivatives of the expectation. Such problems arises with many probabilistic models with noises or latent variables. The paper proposes a new gradient estimator of low variance applicable in certain scenarios, in particular it allows training of generative models in which observations and/or latent variables are discrete. 
The submission clearly improves the state-of-the-art, experimentally demonstrates the method on several problems comparing with the alternative techniques. In what concerns the optimization, the method achieves a better objective value much faster, confirming that it is a lower variance gradient estimator. 
The clarity of the presentation (in particular the description of when the method is applicable) and the technical correctness of the paper are somewhat lacking. In terms of applicability, it seems that many cases where discrete latent variables would be really interesting are not covered (e.g. sigmoid belief networks); the paper demonstrates experiments with discrete images (binary or 4-bit) not particularly motivated in my opinion. It also contains lots of additional technical details and experiments in the appendix, which I unfortunately did not review.
* Clarity
In the abstract the paper promises more than it delivers. Many problems can be cast as optimizing an expectation-based objective. The result does not at all apply to all of them. The reparameterization trick does not apply to all continuous random variables, only to such that the reparameterization satisfies certain smoothness conditions. Discrete variables are supported by the method only in the case that the distribution factors over all discrete variables conditionally on any additional “continuous variables” (to which the reparameterization trick is applicable). This very much limits the utility of the method. In particular it is not applicable to learning e.g. sigmoid belief networks [Neal, 92] (with conditional Bernoulli units) and many other problems. 
“reparametrizable distributions”
A Bernoulli(p) random variable is discrete, yet it is reparametrizable as [Z>p] with Z following standard logistic distribution, whose density and cdf is smooth. 
Because of the above many discussions about discrete vs. continuous variables are missleading.
Section 2. The notation of the true distribution as “q” the model as p and the approximate posterior of the model as “q” again is inconsistent. I find the background on ELBO and GANs unnecessary occluding the clarity at this point. For the purpose of introduction, it might be better to give examples of expectation objectives such as: 
- dropout: q is the distribution of NN outputs given the input image and integrating out latent dropout noises, gamma are parameters of this NN.
- VAE, GAN: q is the generative model defined as a mapping of a standard multivariate normal distribution by a NN.
- sigmoid belief networks: q is a Bayesian network where each conditional distribution is a logistic regression model.
Then to state to which of these cases the results of the paper are applicable, allow for an improvement of the variance and at what additional computational cost (considering the cost of evaluating the discrete derivatives).
Section 3.
Contrary to the discussion, there are examples of non-negative distributions to which the reparameterization trick can be applied, including log-Normal and Gamma distributions.
Method:
In the case when Rep trick is applicable, is it identical to GO? The difference seems to be only in that the mapping tau may be different from Q^-1. However, this only affects the method of drawing the samples from a fixed known distribution and should have no more effect on the results than say a choice of a pseudo-random number generator. Yet, in Fig.1 some difference is observed between the methods, why is that so?
Sec 7.1
“We adopt the sticking approach hereafter”. Does it mean it is applied with all experiments with GO?
* Related Work
The state of the art allows combining differentiable and non-differentiable pieces of computation:
[Schulman, J., Heess, N., Weber, T., Abbeel, P.: Gradient estimation using stochastic computation graphs.]
I believe it should be discussed in related work. Limitations / where the proposed method brings an improvement should be highlighted.
* Technical Correctness
Equations (5) and (6) require a theorem of differentiating under integral (expectation), such as Leibnitz rule, which in case of (6) requires q_gamma(y)f(y) to be continuous in y and q_gamma(y) continuously differentiable in gamma.
Equation (7) (integration by parts) holds only with some additional requires on f.
Theorem 1 does not take account for the above conditions.