Good Semi-supervised Learning That Requires a Bad GAN
Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time. Theoretically we show that given the discriminator objective, good semisupervised learning indeed requires a bad generator, and propose the definition of a preferred generator. Empirically, we derive a novel formulation based on our analysis that substantially improves over feature matching GANs, obtaining state-of-the-art results on multiple benchmark datasets 2 .
After reading the rebuttal I changed my score to 7.  Overall it is an interesting paper with an interesting idea. Although the theoretical contributions are emphasized I find the empirical findings more appealing. The theory presented in the paper is not convincing (input versus feature, convexity etc). 
I think the link to classical semi-supervised learning and the cluster assumption should be emphasized, and the * low density assumption on the boundary* as explained in this paper :
Semi-Supervised Classification by Low Density Separation
Olivier Chapelle, Alexander Zien
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.76.5826&rep=rep1&type=pdf
 
I am changing my review to 7, and I hope that the authors will put their contribution in the context of known work in semi-supervised learning , that the boundary of separation  should lie in the low density regions . This will put the paper better in context.
-----
This paper provides an analysis of how GAN helps in  semi supervised learning (in the “K+1” setting of [15]). The paper outlines some assumptions under which it is shown that a complement generator is needed to improve the accuracy of the supervised task at hand. Meaning that the generator needs to  target low densities areas in the input space. 
Using this idea an algorithm is given that combines that feature matching criterium , with a density estimation (using a pixel CNN ) under which the generator targets low density areas of the fixed estimated model (for a given threshold of log proba,). 
Other entropy regularizers are added to encourage diversity in the generator. Positive empirical results are reported.
 
Understanding GANs in the semi supervised setting and improving  it  is  an important problem, however the paper has many caveats:
- while the analysis is done in the feature space under a lot of assumptions, the method proposed is in the input space , which gives a big mismatch between the analysis and the proposed method . Convexity that is used all over the proofs is not any more valid.
-  the KL expression (line 238 ) is wrong:  the term assumed to be constant is not constant. It is equal to ‘-log(C) P(x sim p_g, p(x)<=epsilon)’, this term should be optimized as well.  Some other way to come up at the objective presented need to be developed. 
Maybe just motivating the minimization of the cross entropy term, and adding a cross entropy regularization?
- From the analysis and the  2D example  with uniform sampling off the manifold, the paper seems to suggest that the generator should supply samples outside the manifold, in a sense it reinforces the boundaries of the classifier by providing only negative samples. The truth is in between: the generator should not provides too strong samples (very realistic, easy to classify as a class ) , nor too weak samples that   are easily labeled as fake. it should be portion on the manifolds to reinforce the positive , and portions outside  to reinforce the negative. 
A more realistic setup may be probabilistic where the assumption are assumed to hold with probability 1- delta as off the manifold, and delta on the manifold, although corollary 1 is not proven and hard to understand how it could hold , it seems not realistic to me .
Balancing this delta seem to be crucial and seems inline with the analysis of https://arxiv.org/pdf/1705.08850.pdf 
 
- Ablation study:  Performance of the method  in section 6 for many values of epsilon values would illustrate the discussion above, and should be reported, also a justification of the 10 quantile should be provided.
-Overall using a density estimation (Pixel CNN) and entropic regularizer with another estimator seems a bit adhoc and not satisfying, wonder if the authors have other thoughts to avoid those estimators while encouraging a form of  'low density sampling’ while using only implicit modeling. or  hybrid implicit/ prescribed modeling while maintaining an end to end training?
Minor comments:
- the loss is L_supervised + L_unsupervised have you experimented with balancing the two terms with a penalty term lambda, how does this  balancing interact with the threshold epsilon?
- In lines '162 and 163'  This builds a connection with ... [2,25] which leverage "the low density boundary assumption". I don't understand this comment , can you elaborate more on what this assumption in laplacian regularization for instance?
- line 178 typo : 'the a  bounded' 
- corollary one : the proof is not provided. I don't understand how those claims are true uniformly over all X, and what does it means |G| goes to infinity, you may need  stronger assumptions here....