Discrete latent-variable models, while applicable in a variety of settings, can often be difficult to learn. Sampling discrete latent variables can result in highvariance gradient estimators for two primary reasons: 1. branching on the samples within the model, and 2. the lack of a pathwise derivative for the samples. While current state-of-the-art methods employ control-variate schemes for the former and continuous-relaxation methods for the latter, their utility is limited by the complexities of implementing and training effective control-variate schemes and the necessity of evaluating (potentially exponentially) many branch paths in the model. Here, we revisit the reweighted wake-sleep (RWS) (Bornschein & Bengio, 2015) algorithm, and through extensive evaluations, show that it circumvents both these issues, outperforming current state-of-the-art methods in learning discrete latent-variable models. Moreover, we observe that, unlike the importance weighted autoencoder, RWS learns better models and inference networks with increasing numbers of particles, and that its benefits extend to continuous latentvariable models as well. Our results suggest that RWS is a competitive, often preferable, alternative for learning deep generative models.
Main idea:
This paper studies a problem of the importance weighted autoencoder (IWAE) pointed out by  Rainforth 18, that is, tighter lower bounds arising from increasing the number of particles improve the learning of the generative model, but worsen the learning of the inference network. The authors show that the reweighted wake-sleep algorithm (RWS) doesn't suffer from this issue. Moreover, as an alternative to control variate scheme and reparameterization trick, RWS doesn't suffer from high variance gradients, thus it is particularly useful for discrete latent variable models.   
To support the claim, they conduct three experiments: 1) on ATTEND, INFER, REPEAT, a generative model with both discrete and continuous latent variables; 2) on MNIST with a continuous latent variable model; 3) on a synthetic GMM.
Clarity issues:
1. "branching" has been used many times, but AFAIK, this seems not a standard terminology. What do "branching on the samples", "conditional branching", "branching paths" mean?
2. zero-forcing failure mode and delta-WW: I find this part difficult to follow. For example, the following sentence 
"the inference network q(z|x) becomes the posterior for this model which, in this model, also has support at most {0, . . . , 9} for all x". 
However, this failure mode seems an interesting finding, and since delta-WW outperforms other methods, it deserves a better introduction. 
Questions:
1. In Fig 1 (right), how do you estimate KL(q(z|x) || p(z|x))?
2. In Sec 4.2, why do you say IWAE learns a better model only up to a point (K = 128) and suffers from diminishing returns afterwards?  
3. In Fig 4, why WS doesn't achieve a better performance when K increasing?
Experiments:
1. Since the motivating story is about discrete latent variable models, better baselines should be compared, e.g. RBM, DVAE, DVAE++, VQ-VAE etc. 
2. All experiments were on either on MNIST or synthetic data, at least one large scale experiment on discrete data should be made to verify the performance of RWS.