We introduce the deep abstaining classifier -a deep neural network trained with a novel loss function that provides an abstention option during training. This allows the DNN to abstain on confusing or difficult-to-learn examples while improving performance on the non-abstained samples. We show that such deep abstaining classifiers can: (i) learn representations for structured noise -where noisy training labels or confusing examples are correlated with underlying features -and then learn to abstain based on such features; (ii) enable robust learning in the presence of arbitrary or unstructured noise by identifying noisy samples; and (iii) be used as an effective out-of-category detector that learns to reliably abstain when presented with samples from unknown classes. We provide analytical results on loss function behavior that enable automatic tuning of accuracy and coverage, and demonstrate the utility of the deep abstaining classifier using multiple image benchmarks, Results indicate significant improvement in learning in the presence of label noise.
This paper formulates a new deep method called deep abstaining classifer. Their main idea is to introduce a new modified loss function that utilizes an absention output allowing the DNN to learn when abstention is a better option. The core idea resemble KWIK framework [1], which has been theoretical justified.
Pros:
1. The authors find a new direction for learning with noisy labels. Based on Eq. (1) (the modified loss), the propose \alpha auto-tuning algorithm, which is relatively novel. 
2. The authors perform numerical experiments to demonstrate the efficacy of their framework. And their experimental result support their previous claims.
For example, they conduct experiments on CIFAR-10 and CIFAR-100. Besides, they conduct experiments on open-world detection dataset.
Cons:
We have three questions in the following.
1. Clarity: in Section 3, the author claim real-world data is corrupted in some non-arbitrary manner. However, in practice, it is really hard to reason the corrpution procedure for agnostic noisy dataset like Clothing1M [2]. The authors are encouraged to explain this point more.
2. Related works: In deep learning with noisy labels, there are three main directions, including small-loss trick [3], estimating noise transition matrix [4,5], and explicit and implicit regularization [6]. I would appreciate if the authors can survey and compare more baselines in their paper.
3. Experiment: 
3.1 Baselines: For noisy labels, the author should compare with [7] directly, which is highly related to your work. Namely, designing new loss function can overcome the issue of noisy labels. Without this comparison, the reported result has less impact. Moreover, the authors should add MentorNet [2] as a baseline https://github.com/google/mentornet
3.2 Datasets: For datasets, I think the author should first compare their methods on symmetric and aysmmetric noisy data. Besides, the authors are encouraged to conduct 1 NLP dataset.
References:
[1] L. Li, M. Littman, and T. Walsh. Knows what it knows: a framework for self-aware learning. In ICML, 2008.
[2] T. Xiao, T. Xia, Y. Yang, C. Huang, and X. Wang. Learning from massive noisy labeled data for image classification. In CVPR, 2015.
[3] L. Jiang, Z. Zhou, T. Leung, L. Li, and L. Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, 2018.
[4] G. Patrini, A. Rozza, A. Menon, R. Nock, and L. Qu. Making deep neural networks robust to label noise: A loss correction approach. In CVPR, 2017.
[5] J. Goldberger and E. Ben-Reuven. Training deep neural-networks using a noise adaptation layer. In ICLR, 2017.
[6] T. Miyato, S. Maeda, M. Koyama, and S. Ishii. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. ICLR, 2016.
[7] Z. Zhang and M. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NIPS, 2018.