Towards Resisting Large Data Variations via Introspective Learning
Learning deep networks which can resist large variations between training andtesting data is essential to build accurate and robust image classifiers. Towardsthis end, a typical strategy is to apply data augmentation to enlarge the trainingset. However, standard data augmentation is essentially a brute-force strategywhich is inefficient, as it performs all the pre-defined transformations to everytraining sample. In this paper, we propose a principled approach to train networkswith significantly improved resistance to large variations between training andtesting data. This is achieved by embedding a learnable transformation moduleinto the introspective networks (Jin et al., 2017; Lazarow et al., 2017; Lee et al.,2018), which is a convolutional neural network (CNN) classifier empowered withgenerative capabilities. Our approach alternatively synthesizes pseudo-negativesamples with learned transformations and enhances the classifier by retraining itwith synthesized samples. Experimental results verify that our approach signif-icantly improves the ability of deep networks to resist large variations betweentraining and testing data and achieves classification accuracy improvements onseveral benchmark datasets, including MNIST, affNIST, SVHN and CIFAR-10.
 This paper proposes a way to use generative modeling (specifically, introspective learning) to handle data variations at test time. While the proposed idea is interesting, many of the core ideas seem to have been proposed in earlier work (Jin et al., 2017; Lazarow et al., 2017; Lee et al., 2018) and the novel contributions of this work are not clearly explained. It would be useful to split Section 3 into background section which reviews prior work and a section which describes the proposed idea and how it differs from prior work. Experiments: - The performance numbers in Table 1 seem quite far from the earlier published results. For instance, the ICN paper reports 6.5% test error on CIFAR and 1.9% test error on SVHN, which are much lower than the results reported in this paper https://papers.nips.cc/paper/6684-introspective-classification-with-convolutional-nets.pdf. It would be better to similar architectures as the original paper for a fair comparison. Other comments: - Section 3: it might be worth making a distinction between the two notions in which generative versus discriminative methods are discussed (i) using a discriminative binary classifier within a generative model and (ii) using the classifier that predicts class labels (e.g. the 10 classes in MNIST). - The extension to multi-class problems is only briefly discussed in page 4. Which version (series of one-vs-all or a single CNN classifier) is used in the experiments? What*s the computational complexity and how would it scale with the number of classes? - The semi-supervised learning results only discuss GAN-based methods, but there are a lot of VAE-based methods (cf. https://arxiv.org/abs/1406.5298) that would be worth discussing too - I think it’s a bit confusing to label the methods "WGAN-GP", "DCGAN" as these methods uses both CNN for classifier in addition to the generative part.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Towards Resisting Large Data Variations via Introspective Learning
Learning deep networks which can resist large variations between training andtesting data is essential to build accurate and robust image classifiers. Towardsthis end, a typical strategy is to apply data augmentation to enlarge the trainingset. However, standard data augmentation is essentially a brute-force strategywhich is inefficient, as it performs all the pre-defined transformations to everytraining sample. In this paper, we propose a principled approach to train networkswith significantly improved resistance to large variations between training andtesting data. This is achieved by embedding a learnable transformation moduleinto the introspective networks (Jin et al., 2017; Lazarow et al., 2017; Lee et al.,2018), which is a convolutional neural network (CNN) classifier empowered withgenerative capabilities. Our approach alternatively synthesizes pseudo-negativesamples with learned transformations and enhances the classifier by retraining itwith synthesized samples. Experimental results verify that our approach signif-icantly improves the ability of deep networks to resist large variations betweentraining and testing data and achieves classification accuracy improvements onseveral benchmark datasets, including MNIST, affNIST, SVHN and CIFAR-10.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes a way to use generative modeling (specifically, introspective learning) to handle data variations at test time. While the proposed idea is interesting, many of the core ideas seem to have been proposed in earlier work (Jin et al., 2017; Lazarow et al., 2017; Lee et al., 2018) and the novel contributions of this work are not clearly explained. It would be useful to split Section 3 into background section which reviews prior work and a section which describes the proposed idea and how it differs from prior work. Experiments: - The performance numbers in Table 1 seem quite far from the earlier published results. For instance, the ICN paper reports 6.5% test error on CIFAR and 1.9% test error on SVHN, which are much lower than the results reported in this paper https://papers.nips.cc/paper/6684-introspective-classification-with-convolutional-nets.pdf. It would be better to similar architectures as the original paper for a fair comparison. Other comments: - Section 3: it might be worth making a distinction between the two notions in which generative versus discriminative methods are discussed (i) using a discriminative binary classifier within a generative model and (ii) using the classifier that predicts class labels (e.g. the 10 classes in MNIST). - The extension to multi-class problems is only briefly discussed in page 4. Which version (series of one-vs-all or a single CNN classifier) is used in the experiments? What*s the computational complexity and how would it scale with the number of classes? - The semi-supervised learning results only discuss GAN-based methods, but there are a lot of VAE-based methods (cf. https://arxiv.org/abs/1406.5298) that would be worth discussing too - I think it’s a bit confusing to label the methods "WGAN-GP", "DCGAN" as these methods uses both CNN for classifier in addition to the generative part.
assistant
 This paper proposes a way to use generative modeling (specifically, introspective learning) to handle data variations at test time. While the proposed idea is interesting, many of the core ideas seem to have been proposed in earlier work (Jin et al., 2017; Lazarow et al., 2017; Lee et al., 2018) and the novel contributions of this work are not clearly explained. It would be useful to split Section 3 into background section which reviews prior work and a section which describes the proposed idea and how it differs from prior work. Experiments: - The performance numbers in Table 1 seem quite far from the earlier published results. For instance, the ICN paper reports 6.5% test error on CIFAR and 1.9% test error on SVHN, which are much lower than the results reported in this paper https://papers.nips.cc/paper/6684-introspective-classification-with-convolutional-nets.pdf. It would be better to similar architectures as the original paper for a fair comparison. Other comments: - Section 3: it might be worth making a distinction between the two notions in which generative versus discriminative methods are discussed (i) using a discriminative binary classifier within a generative model and (ii) using the classifier that predicts class labels (e.g. the 10 classes in MNIST). - The extension to multi-class problems is only briefly discussed in page 4. Which version (series of one-vs-all or a single CNN classifier) is used in the experiments? What*s the computational complexity and how would it scale with the number of classes? - The semi-supervised learning results only discuss GAN-based methods, but there are a lot of VAE-based methods (cf. https://arxiv.org/abs/1406.5298) that would be worth discussing too - I think it’s a bit confusing to label the methods "WGAN-GP", "DCGAN" as these methods uses both CNN for classifier in addition to the generative part.