Label Cleaning with Likelihood Ratio Test
To collect large scale annotated data, it is inevitable to introduce label noise, i.e., incorrect class labels. A major challenge is to develop robust deep learning models that achieve high test performance despite training set label noise. We introduce a novel approach that directly cleans labels in order to train a high quality model. Our method leverages statistical principles to correct data labels and has a theoretical guarantee of the correctness. In particular, we use a likelihood ratio test(LRT) to flip the labels of training data. We prove that our LRT label correction algorithm is guaranteed to flip the label so it is consistent with the true Bayesian optimal decision rule with high probability. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.
Label noise widely exists in the large-scale data sets. This paper proposes a novel approach that directly cleans labels in order to train a high quality model. The proposed method leverages statistical principles to correct data labels and has a theoretical guarantee of the correctness. In particular, a likelihood ratio test (LRT) to flip the labels of training data is used, and it can prove that the LRT label correction algorithm is guaranteed to flip the label so it is consistent with the true Bayesian optimal classifier with high probability. The experimental results on several benchmark data sets show that the proposed method is promising. Overall, this paper could be a significant algorithmic contribution. But I also have some minor concerns: [1] The theoretical analysis and the experimental results are both well organized in the paper. How about the time complexity of the proposed method. If the authors can show the time cost in the paper, I will much more agree with the paper. [2] In the experimental parts, the convergence curve of the proposed method during the training epochs may be better to prove the theoretical analysis. [3]The details of the compared methods should be given, and it will be better to give the results without any noise labels. In this way, the confidence of the paper will be further improved.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Label Cleaning with Likelihood Ratio Test
To collect large scale annotated data, it is inevitable to introduce label noise, i.e., incorrect class labels. A major challenge is to develop robust deep learning models that achieve high test performance despite training set label noise. We introduce a novel approach that directly cleans labels in order to train a high quality model. Our method leverages statistical principles to correct data labels and has a theoretical guarantee of the correctness. In particular, we use a likelihood ratio test(LRT) to flip the labels of training data. We prove that our LRT label correction algorithm is guaranteed to flip the label so it is consistent with the true Bayesian optimal decision rule with high probability. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Label noise widely exists in the large-scale data sets. This paper proposes a novel approach that directly cleans labels in order to train a high quality model. The proposed method leverages statistical principles to correct data labels and has a theoretical guarantee of the correctness. In particular, a likelihood ratio test (LRT) to flip the labels of training data is used, and it can prove that the LRT label correction algorithm is guaranteed to flip the label so it is consistent with the true Bayesian optimal classifier with high probability. The experimental results on several benchmark data sets show that the proposed method is promising. Overall, this paper could be a significant algorithmic contribution. But I also have some minor concerns: [1] The theoretical analysis and the experimental results are both well organized in the paper. How about the time complexity of the proposed method. If the authors can show the time cost in the paper, I will much more agree with the paper. [2] In the experimental parts, the convergence curve of the proposed method during the training epochs may be better to prove the theoretical analysis. [3]The details of the compared methods should be given, and it will be better to give the results without any noise labels. In this way, the confidence of the paper will be further improved.
assistant
Label noise widely exists in the large-scale data sets. This paper proposes a novel approach that directly cleans labels in order to train a high quality model. The proposed method leverages statistical principles to correct data labels and has a theoretical guarantee of the correctness. In particular, a likelihood ratio test (LRT) to flip the labels of training data is used, and it can prove that the LRT label correction algorithm is guaranteed to flip the label so it is consistent with the true Bayesian optimal classifier with high probability. The experimental results on several benchmark data sets show that the proposed method is promising. Overall, this paper could be a significant algorithmic contribution. But I also have some minor concerns: [1] The theoretical analysis and the experimental results are both well organized in the paper. How about the time complexity of the proposed method. If the authors can show the time cost in the paper, I will much more agree with the paper. [2] In the experimental parts, the convergence curve of the proposed method during the training epochs may be better to prove the theoretical analysis. [3]The details of the compared methods should be given, and it will be better to give the results without any noise labels. In this way, the confidence of the paper will be further improved.