In semi-supervised learning, Bad GAN approach is one of the most attractive method due to the intuitional simplicity and powerful performances. Bad GAN learns a classifier with bad samples distributed on complement of the support of the input data. But Bad GAN needs additional architectures, a generator and a density estimation model, which involves huge computation and memory consumption cost. VAT is another good semi-supervised learning algorithm, which utilizes unlabeled data to improve the invariance of the classifier with respect to perturbation of inputs. In this study, we propose a new method by combining the ideas of Bad GAN and VAT. The proposed method generates bad samples of high-quality by use of the adversarial training used in VAT. We give theoretical explanations why the adversarial training is good at both generating bad samples and semi-supervised learning. An advantage of the proposed method is to achieve the competitive performances with much fewer computations. We demonstrate advantages our method by various experiments with well known benchmark image datasets.
This paper makes the interesting observation that the generative procedure proposed by Bad GAN paper can be replaced by a slightly modified VAT procedure. The reasoning is sound and leverages the intuition that adversarial examples (subject to a sufficiently small perturbation radius) are likely to be closer to a decision boundary than the original sample. 
The paper is generally easy to follow but the presentation could be improved. In particular more could be done to describe the terms in Equation 5. I’m also curious about the behavior of L^true, which is equivalently the fourth term in Eq 1. Even when reading Bad GAN paper, I did not quite understand their claim that this can be correctly interpreted as a conditional entropy term (if they really wanted conditional entropy, they should probably have either done H(p(k|x)) or H(p(k|x, k <= K))). I agree with the authors that the roles of the second and fourth terms overlapped, and I think this is sufficiently interesting to warrant some further elaboration in the paper. I also liked the reminder that power iteration selects a non-unique sign for the first eigenvector (subject to the random vector initialization); I encourage the authors to do an ablation test to convince the reader that “this modification helps to improve convergence speed of the test accuracy.”
The propositions in this paper were, in my opinion, not particularly insightful. While I think it is nice that the authors went through the effort of providing some formalism to the intuition that VAT has a “push decision boundary away from high-density regions”, I’m less sure if propositions 1 and 2 really provides any additional insight the behavior of VAT. Proposition 1 is pretty weak in that it only covers a 2-class logistic regression; it seems obvious that the adversarial perturbation points in the direction toward the decision hyperplane. If the authors could extend this to more general non-linear classifiers (perhaps subject to some assumptions), that would be more interesting. I don’t think Proposition 2 has any real value and recommend its relegation to the appendix.
I think the biggest weakness of this paper is the experiments. Taking Table 1 at face value, the conclusion that FAT is simply competitive with existing approaches suggests that the additional machinery isn’t particularly useful, providing little more than a vanilla VAT. I also think MNIST/SVHN has run its course as good semi-supervised learning benchmarks and would prefer to see such algorithms being scaled to more complex data. The main argument for why FAT should be prefered over VAT comes from Section 6.2. Figure 4 is more interesting, but is complicated by the fact that FAT checks both possible eigenvectors (+/- u) during training, which requires two forward passes in the classifier; did the authors give a similar treatment to VAT? Please show wall-clock time too. Unfortunately the computational efficiency gain seems to only hold true for MNIST/SVHN, but not for CIFAR. I worry that the observed gains will not sustain once we move to more complicated datasets.
Pros:
+ Simple and clean proposal
+ Easy to read
Cons:
- Limited insight
- Weak experiments