Reducing Overconfident Errors outside the Known Distribution
Intuitively, unfamiliarity should lead to lack of confidence. In reality, current algorithms often make highly confident yet wrong predictions when faced with unexpected test samples from an unknown distribution different from training. Unlike domain adaptation methods, we cannot gather an "unexpected dataset" prior to test, and unlike novelty detection methods, a best-effort original task prediction is still expected. We compare a number of methods from related fields such as calibration and epistemic uncertainty modeling, as well as two proposed methods that reduce overconfident errors of samples from an unknown novel distribution without drastically increasing evaluation time: (1) G-distillation, training an ensemble of classifiers and then distill into a single model using both labeled and unlabeled examples, or (2) NCR, reducing prediction confidence based on its novelty detection score. Experimentally, we investigate the overconfidence problem and evaluate our solution by creating "familiar" and "novel" test splits, where "familiar" are identically distributed with training and "novel" are not. We discover that calibrating using temperature scaling on familiar data is the best single-model method for improving novel confidence, followed by our proposed methods. In addition, some methods* NLL performance are roughly equivalent to a regularly trained model with certain degree of smoothing. Calibrating can also reduce confident errors, for example, in gender recognition by 95% on demographic groups different from the training data.
 The paper proposes two ideas for reducing overconfident wrong predictions: - Method 1: “G-distillation” of an ensemble with extra unsupervised data - Method 2: Novelty Confidence reduction (NCR) using novelty detector The paper is well-written and was a pleasure to read. In particular, I really enjoyed reading the introduction and related work. My main concern is that some of the contributions claimed were already shown in previous work (see method 1 below for details), and the novelty feels a bit limited. That said, I like the simplicity of the method and think that the extensive experiments on a variety of datasets and architectures is useful to the community. Method 1: - The paper claims “Draw attention to a counter-intuitive yet important problem of highly confident wrong predictions when samples are drawn from a unknown distribution that is different than training” as one of the contributions. Note that previous work has already shown that single models are overconfident on unknown classes and ensembles are less overconfident, e.g. see Section 3.5 of the paper: Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles https://arxiv.org/pdf/1612.01474.pdf - If I understand correctly, the key difference is that the proposed method 1 also uses ensemble prediction on unlabeled data for distillation, which could make the distilled model more robust. Ensembling on unlabeled data for robustness does seem novel to me, however, the text needs to be updated to clarify the novelty. Method 2: - By off-the-shelf, do you mean a pre-trained network released by ODIN? Or did you train ODIN-based novelty detector on your dataset? - There was a recent paper that proposed to reduce confidence on novel inputs, which might be worth discussing: Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors https://arxiv.org/pdf/1807.09289.pdf Minor issues: - Figures 3,4 are a bit small and hard to see
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Reducing Overconfident Errors outside the Known Distribution
Intuitively, unfamiliarity should lead to lack of confidence. In reality, current algorithms often make highly confident yet wrong predictions when faced with unexpected test samples from an unknown distribution different from training. Unlike domain adaptation methods, we cannot gather an "unexpected dataset" prior to test, and unlike novelty detection methods, a best-effort original task prediction is still expected. We compare a number of methods from related fields such as calibration and epistemic uncertainty modeling, as well as two proposed methods that reduce overconfident errors of samples from an unknown novel distribution without drastically increasing evaluation time: (1) G-distillation, training an ensemble of classifiers and then distill into a single model using both labeled and unlabeled examples, or (2) NCR, reducing prediction confidence based on its novelty detection score. Experimentally, we investigate the overconfidence problem and evaluate our solution by creating "familiar" and "novel" test splits, where "familiar" are identically distributed with training and "novel" are not. We discover that calibrating using temperature scaling on familiar data is the best single-model method for improving novel confidence, followed by our proposed methods. In addition, some methods* NLL performance are roughly equivalent to a regularly trained model with certain degree of smoothing. Calibrating can also reduce confident errors, for example, in gender recognition by 95% on demographic groups different from the training data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper proposes two ideas for reducing overconfident wrong predictions: - Method 1: “G-distillation” of an ensemble with extra unsupervised data - Method 2: Novelty Confidence reduction (NCR) using novelty detector The paper is well-written and was a pleasure to read. In particular, I really enjoyed reading the introduction and related work. My main concern is that some of the contributions claimed were already shown in previous work (see method 1 below for details), and the novelty feels a bit limited. That said, I like the simplicity of the method and think that the extensive experiments on a variety of datasets and architectures is useful to the community. Method 1: - The paper claims “Draw attention to a counter-intuitive yet important problem of highly confident wrong predictions when samples are drawn from a unknown distribution that is different than training” as one of the contributions. Note that previous work has already shown that single models are overconfident on unknown classes and ensembles are less overconfident, e.g. see Section 3.5 of the paper: Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles https://arxiv.org/pdf/1612.01474.pdf - If I understand correctly, the key difference is that the proposed method 1 also uses ensemble prediction on unlabeled data for distillation, which could make the distilled model more robust. Ensembling on unlabeled data for robustness does seem novel to me, however, the text needs to be updated to clarify the novelty. Method 2: - By off-the-shelf, do you mean a pre-trained network released by ODIN? Or did you train ODIN-based novelty detector on your dataset? - There was a recent paper that proposed to reduce confidence on novel inputs, which might be worth discussing: Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors https://arxiv.org/pdf/1807.09289.pdf Minor issues: - Figures 3,4 are a bit small and hard to see
assistant
 The paper proposes two ideas for reducing overconfident wrong predictions: - Method 1: “G-distillation” of an ensemble with extra unsupervised data - Method 2: Novelty Confidence reduction (NCR) using novelty detector The paper is well-written and was a pleasure to read. In particular, I really enjoyed reading the introduction and related work. My main concern is that some of the contributions claimed were already shown in previous work (see method 1 below for details), and the novelty feels a bit limited. That said, I like the simplicity of the method and think that the extensive experiments on a variety of datasets and architectures is useful to the community. Method 1: - The paper claims “Draw attention to a counter-intuitive yet important problem of highly confident wrong predictions when samples are drawn from a unknown distribution that is different than training” as one of the contributions. Note that previous work has already shown that single models are overconfident on unknown classes and ensembles are less overconfident, e.g. see Section 3.5 of the paper: Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles https://arxiv.org/pdf/1612.01474.pdf - If I understand correctly, the key difference is that the proposed method 1 also uses ensemble prediction on unlabeled data for distillation, which could make the distilled model more robust. Ensembling on unlabeled data for robustness does seem novel to me, however, the text needs to be updated to clarify the novelty. Method 2: - By off-the-shelf, do you mean a pre-trained network released by ODIN? Or did you train ODIN-based novelty detector on your dataset? - There was a recent paper that proposed to reduce confidence on novel inputs, which might be worth discussing: Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors https://arxiv.org/pdf/1807.09289.pdf Minor issues: - Figures 3,4 are a bit small and hard to see