Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping. We propose a new regularization method, which enables learning robust classifiers in presence of noisy data. To achieve this goal, we propose a new adversarial regularization scheme based on the Wasserstein distance. Using this distance allows taking into account specific relations between classes by leveraging the geometric properties of the labels space. Our Wasserstein Adversarial Regularization (WAR) encodes a selective regularization, which promotes smoothness of the classifier between some classes, while preserving sufficient complexity of the decision boundary between others. We first discuss how and why adversarial regularization can be used in the context of label noise and then show the effectiveness of our method on five datasets corrupted with noisy labels: in both benchmarks and real datasets, WAR outperforms the state-of-the-art competitors.
Update after rebuttal:
The rebuttal addresses my concerns. The point raised by reviewer 2 regarding other results on Clothing1M is concerning, but the authors' response is reasonable. Moreover, the proposed method is novel and could be complementary with other methods that achieve good results on Clothing1M. I think this paper would generate good discussion, and I recommend acceptance.
----------------------------------------------------------------------------------------
This paper presents a novel approach for dealing with asymmetric label noise. There are in fact two methods proposed, Adversarial Regularization (AR) and Wasserstein Adversarial Regularization (WAR), both of which derive from the intuition that model smoothing (in the sense of virtual adversarial training) acts as label smoothing when there are noisy labels. The authors prove two statements formalizing this notion and demonstrate the efficacy of AR and WAR in an extensive evaluation.
WAR is an iteration on AR that uses the optimal transport distance between categorical distributions to allow incorporating a cost matrix that encodes class similarities. The authors obtain these similarities from word embeddings of class names.
The evaluation demonstrates that the proposed WAR method achieves state-of-the-art results with the largest gains at high noise levels. In no case does it perform substantially worse than competing methods. Moreover, the AR method proposed as an intermediary to WAR also outperforms all prior work at non-zero noise levels, demonstrating the promise of the underlying approach.
The authors also evaluate on Clothing1M and evaluate on a semantic segmentation dataset for which they synthesize structured, realistic label noise. WAR outperforms prior work on these tasks. Moreover, the semantic segmentation task could act as a useful benchmark for future work.
For these reasons, I recommend acceptance.
Minor points:
Comparing the second and third columns in Figure 1, it looks like lambda has a large effect, so it would be good to know how much tuning lambda requires for the main tasks.
In equation 9, shouldn’t there be a beta in front of the entropy term? This does not break the result.
In equation 10, it looks like you’re multiplying both sides by (1 – epsilon), but you drop the (1 – epsilon) on the left side. This does not break the result.