Generative Adversarial Networks are powerful generative models that can model the manifold of natural images. We leverage this property to perform manifold regularization by approximating a variant of the Laplacian norm using a Monte Carlo approximation that is easily computed with the GAN. When incorporated into the semi-supervised feature-matching GAN we achieve state-of-the-art results for semi-supervised learning on CIFAR-10 benchmarks when few labels are used, with a method that is significantly easier to implement than competing methods. We find that manifold regularization improves the quality of generated images, and is affected by the quality of the GAN used to approximate the regularizer.
This paper builds upon the assumption that GANs successfully approximate the data manifold, and uses this assumption to regularize semi-supervised learning process.
The proposed regularization strategy enforces that a discriminator or a given classifier should be invariant to small perturbations on the data manifold z. It is empirically shown that naively enforcing such a constraint by randomly adding noise to z could lead to under-smoothing or over-smoothing in some cases which can harm the final classification performance. Consequently, the proposed regularization technique takes a step of tunable size in the direction of the manifold gradient, which has the effect of smoothing along the direction of the gradient while ignoring its norm.
 
Extensive experiments have been conducted, showing that the proposed approach
outperforms or is comparable with recent state-of-the-art approaches on cifar 10, especially in presence of fewer labelled data points. On SVHN however, the proposed approach fails in comparison with (Kumar et al 2017) but performs better than other approaches.
Furthermore, it has been shown that adding the proposed manifold regularization technique to the training of GAN greatly improves the image quality of generated images (in terms of FID scores and inception scores). Also, by combining the proposed regularizer with a classical supervised classifier (via pre-training a GAN and using it for regularization) decreases classification error by 2 to 3%.
 
Finally, it has also been shown that after training a GAN using the manifold regularization, the algorithm is able to produce similar images giving a low enough perturbation of the data manifold z.
 
Overall, this paper is well written and show significant improvements especially for image generation. However, the novelty is rather limited as similar ideas have been undertaken (e.g., Mescheder et al 2018), but in different contexts. The paper would be improved if the following points are taken into account:
 
A comparison with Graph Convolutional Network based techniques seems appropriate (e.g. Kipf and Welling 2017).
How do the FID/Inception improvements compare to (Mescheder et al 2018)?
It would be interesting to discuss why the FID score for SVHN gets worse in presence of 1000 labels.
Although there is a clear improvement in FID scores for Cifar10. It would be informative to show the generated images w/ and w/o manifold regularization.
More analysis should be provided on why (Kumar et al 2017) perform so well on SVHN.
It should be stated that bold values in tables do not represent best results (as it is usually the case) but rather results for the proposed approach.