We present Cross-View Training (CVT), a simple but effective method for deep semi-supervised learning. On labeled examples, the model is trained with standard cross-entropy loss. On an unlabeled example, the model first performs inference (acting as a "teacher") to produce soft targets. The model then learns from these soft targets (acting as a "student"). We deviate from prior work by adding multiple auxiliary student prediction layers to the model. The input to each auxiliary student layer is a sub-network of the full model that has a restricted view of the input (e.g., only seeing one region of an image). The students can learn from the teacher (the full model) because the teacher sees more of each example. Concurrently, the students improve the quality of the representations used by the teacher as they learn to make predictions with limited data. When combined with Virtual Adversarial Training, CVT improves upon the current state-of-the-art on semi-supervised CIFAR-10 and semi-supervised SVHN. We also apply CVT to train models on five natural language processing tasks using hundreds of millions of sentences of unlabeled data. On all tasks CVT substantially outperforms supervised learning alone, resulting in models that improve upon or are competitive with the current state-of-the-art.
This paper presents a so-called cross-view training for semi-supervised deep models. Experiments were conducted on various data sets and experimental results were reported.
Pros:
* Studying semi-supervised learning techniques for deep models is of practical significance.
Cons:
* The novelty of this paper is marginal. The use of unlabeled data is in fact a self-training process. Leveraging the sub-regions of the image to improve performance is not new and has been widely-studied in image classification and retrieval. 
* The proposed approach suffers from a technical weakness or flaw. For the self-labeled data, the prediction of each view is enforced to be same as the assigned self-labeling. However, since each view related to a sub-region of the image (especially when the model is not so deep), it is less likely for this region to contain the representation of the concepts (e.g., some local region of an image with a horse may exhibit only grass); enforcing the prediction of this view to be the same self-labeled concepts (e.g,“horse”) may drive the prediction away from what it should be ( e..g, it will make the network to predict grass as horse). Such a flaw may affect the final performance of the proposed approach.
* The word “view” in this paper is misleading. The “view” in this paper is corresponding to actually sub-regions in the images
* The experimental results indicate that the proposed approach fails to perform better than the compared baselines in table 2, which reduces the practical significance of the proposed approach.