Training deep neural-networks using a noise adaptation layer
The availability of large datsets has enabled neural networks to achieve impressive recognition results. However, the presence of inaccurate class labels is known to deteriorate the performance of even the best classifiers in a broad range of classification problems. Noisy labels also tend to be more harmful than noisy attributes. When the observed label is noisy, we can view the correct label as a latent random variable and model the noise processes by a communication channel with unknown parameters. Thus we can apply the EM algorithm to find the parameters of both the network and the noise and to estimate the correct label. In this study we present a neural-network approach that optimizes the same likelihood function as optimized by the EM algorithm. The noise is explicitly modeled by an additional softmax layer that connects the correct labels to the noisy ones. This scheme is then extended to the case where the noisy labels are dependent on the features in addition to the correct labels. Experimental results demonstrate that this approach outperforms previous methods.
The paper addressed the erroneous label problem for supervised training. The problem is well formulated and the presented solution is novel. The experimental justification is limited. The effectiveness of the proposed method is hard to gauge, especially how to scale the proposed method to large number of classification targets and whether it is still effective. For example, it would be interesting to see whether the proposed method is better than training with only less but high quality data. From Figure 2, it seems with more data, the proposed method tends to behave very well when the noise fraction is below a threshold and dramatically degrades once passing that threshold. Analysis and justification of this behavior whether it is just by chance or an expected one of the method would be very useful.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Training deep neural-networks using a noise adaptation layer
The availability of large datsets has enabled neural networks to achieve impressive recognition results. However, the presence of inaccurate class labels is known to deteriorate the performance of even the best classifiers in a broad range of classification problems. Noisy labels also tend to be more harmful than noisy attributes. When the observed label is noisy, we can view the correct label as a latent random variable and model the noise processes by a communication channel with unknown parameters. Thus we can apply the EM algorithm to find the parameters of both the network and the noise and to estimate the correct label. In this study we present a neural-network approach that optimizes the same likelihood function as optimized by the EM algorithm. The noise is explicitly modeled by an additional softmax layer that connects the correct labels to the noisy ones. This scheme is then extended to the case where the noisy labels are dependent on the features in addition to the correct labels. Experimental results demonstrate that this approach outperforms previous methods.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper addressed the erroneous label problem for supervised training. The problem is well formulated and the presented solution is novel. The experimental justification is limited. The effectiveness of the proposed method is hard to gauge, especially how to scale the proposed method to large number of classification targets and whether it is still effective. For example, it would be interesting to see whether the proposed method is better than training with only less but high quality data. From Figure 2, it seems with more data, the proposed method tends to behave very well when the noise fraction is below a threshold and dramatically degrades once passing that threshold. Analysis and justification of this behavior whether it is just by chance or an expected one of the method would be very useful.
assistant
The paper addressed the erroneous label problem for supervised training. The problem is well formulated and the presented solution is novel. The experimental justification is limited. The effectiveness of the proposed method is hard to gauge, especially how to scale the proposed method to large number of classification targets and whether it is still effective. For example, it would be interesting to see whether the proposed method is better than training with only less but high quality data. From Figure 2, it seems with more data, the proposed method tends to behave very well when the noise fraction is below a threshold and dramatically degrades once passing that threshold. Analysis and justification of this behavior whether it is just by chance or an expected one of the method would be very useful.