Noisy labels are very common in real-world training data, which lead to poor generalization on test data because of overfitting to the noisy labels. In this paper, we claim that such overfitting can be avoided by "early stopping" training a deep neural network before the noisy labels are severely memorized. Then, we resume training the early stopped network using a "maximal safe set," which maintains a collection of almost certainly true-labeled samples at each epoch since the early stop point. Putting them all together, our novel two-phase training method, called Prestopping, realizes noise-free training under any type of label noise for practical use. Extensive experiments using four image benchmark data sets verify that our method significantly outperforms four state-of-the-art methods in test error by 0.4-8.2 percent points under existence of real-world noise.
This paper proposes a training strategy for robustness against label noise. The training strategy is simple and straightforward. The neural network will first be trained on the entire dataset with all the noisy labels. After obtaining the network with lowest validation error, the network will be used to make a prediciton on the original training set and select a subset of it to construct a maximal safe set. Finally, the network will be findtuned on this maximal safe set. The training strategy is very similar to tradictional  self-training in semi-superivsed learning and co-training for domain adaptation ([Co-training for domain adaptation, NIPS 2011]), except that the proposed prestopping only iterate the procedure once.
The paper discusses two important questions for the method: (1) when to early stop the training; (2) how to constuct a maximal safe set. The authors' responses to these questions are very natual but less interesting. Using the lowest validation error to early stop the training could be suboptimal, since the small validation set can not fully capture the data distribution and could make the network empirically overfit to this validation set. The criterion to contruct a maxial safe set is also conventional, and is similar to what a number of papers are doing, for example,
[1] Co-training for domain adaptation, NIPS 2011
[2] Self-ensembling for visual domain adaptation, ICLR 2018
[3] A dirt-t approach to unsupervised domain adaptation, ICLR 2018
[4] Iterative learning with open-set noisy labels, CVPR 2018
In experiments, the results are not very surprising. There are some baselines that adopt a similar (iterative) pipeline (learning the network - selecting a subset of the training samples - re-learning the network):
[1] Iterative Learning with Open-set Noisy Labels, CVPR 2018
[2] Dimensionality-Driven Learning with Noisy Labels, ICML 2018
[3] Symmetric Cross Entropy for Robust Learning with Noisy Labels, ICCV 2019
The authors can consider to compare to some of these baselines, especially [1] and [2]. The difference between the paper and [1,2] is basically the criterion to construct the maximal safe subset.
Besides, I suggest the authors to conduct large-scale experiments on ImageNet or even a subset of ImageNet, since the difficulty of detecting label noise is much higher when the resolution of images become bigger. CIFAR-10 and CIFAR-100 only contain 32x32 images, which is far less challenging.
Overall, I think the paper is well written, the idea is clearly presented, and the experiments also seem convinceing. However, the contribution of this paper is very incremental.