We exploit a recently derived inversion scheme for arbitrary deep neural networks to develop a new semi-supervised learning framework that applies to a wide range of systems and problems. The approach reaches current state-of-the-art methods on MNIST and provides reasonable performances on SVHN and CIFAR10. Through the introduced method, residual networks are for the first time applied to semi-supervised tasks. Experiments with one-dimensional signals highlight the generality of the method. Importantly, our approach is simple, efficient, and requires no change in the deep network architecture.
In summary, the paper is based on a recent work Balestriero & Baraniuk 2017 to do semi-supervised learning. In Balestriero & Baraniuk, it is shown that any DNN can be approximated via a linear spline and hence can be inverted to produce the "reconstruction" of the input, which can be naturally used to do unsupervised or semi-supervised learning. This paper proposes to use automatic differentiation to compute the inverse function efficiently. The idea seems interesting. However, I think there are several main drawbacks, detailed as follows:
1. The paper lacks a coherent and complete review of the semi-supervised deep learning. Herewith some important missing papers, which are the previous or current state-of-the-art.
[1] Laine S, Aila T. Temporal Ensembling for Semi-Supervised Learning[J]. arXiv preprint arXiv:1610.02242, ICLR 2016.
[2] Li C, Xu K, Zhu J, et al. Triple Generative Adversarial Nets[J]. arXiv preprint arXiv:1703.02291, NIPS 2017.
[3] Dai Z, Yang Z, Yang F, et al. Good Semi-supervised Learning that Requires a Bad GAN[J]. arXiv preprint arXiv:1705.09783, NIPS 2017.
Besides, some papers should be mentioned in the related work such as Kingma et. al. 2014. I'm not an expert of the network inversion and not sure whether the related work of this part is sufficient or not.
2. The motivation is not sufficient and not well supported. 
As stated in the introduction, the authors think there are several drawbacks of existing methods including "training instability, lack of topology generalization and computational complexity." Based on my knowledge, there are two main families of semi-supervised deep learning methods, classified by depending on deep generative models or not.  The generative approaches based on VAEs and GANs are time consuming, but according to my experience, the training of VAE-based methods are stable and the topology generalization ability of such methods are good. Besides, the feed-forward approaches including [1] mentioned above are efficient and not too sensitive with respect to the network architectures.  Overall, I think the drawbacks mentioned in the paper are not common in existing methods and I do not see clear benefits of the proposed method. Again, I strongly suggest the authors to provide a complete review of the literature.
Further, please explicitly support your claim via experiments. For instance, the proposed method should be compared  with the discriminative approaches including VAT and [1] in terms of the training efficiency. It's not fair to say GAN-based methods require more training time because these methods can do generation and style-class disentanglement while the proposed method cannot.
3. The experimental results are not so convincing. 
First, please systematically compare your methods with existing methods on the widely adopted benchmarks including MNIST with 20, 100 labels and SVHN with 500, 1000 labels and CIFAR10 with 4000 labels. It is not safe to say the proposed method is the state-of-the-art by only showing the results in one setting.
Second, please report the results of the proposed method with comparable architectures used in previous methods and state clearly the number of parameters in each model. Resnet is powerful but previous methods did not use that.
Last, show the sensitive results of the proposed method by tuning alpha and beta. For instance, please show what is the actual contribution of the proposed reconstruction loss to the classification accuracy with the other losses existing or not?
I think the quality of the paper should be further improved by addressing these problems and currently it should be rejected.