Training-Free Uncertainty Estimation for Neural Networks
Uncertainty estimation is an essential step in the evaluation of the robustness for deep learning models in computer vision, especially when applied in risk-sensitive areas. However, most state-of-the-art deep learning models either fail to obtain uncertainty estimation or need significant modification (e.g., formulating a proper Bayesian treatment) to obtain it. None of the previous methods are able to take an arbitrary model off the shelf and generate uncertainty estimation without retraining or redesigning it. To address this gap, we perform the first systematic exploration into training-free uncertainty estimation. We propose three simple and scalable methods to analyze the variance of output from a trained network under tolerable perturbations: infer-transformation, infer-noise, and infer-dropout. They operate solely during inference, without the need to re-train, re-design, or fine-tune the model, as typically required by other state-of-the-art uncertainty estimation methods. Surprisingly, even without involving such perturbations in training, our methods produce comparable or even better uncertainty estimation when compared to other training-required state-of-the-art methods. Last but not least, we demonstrate that the uncertainty from our proposed methods can be used to improve the neural network training.
This paper addresses the problem of deriving better predictive uncertainty estimates from pre-trained models. As the paper notes in the introduction, this is an important problem for machine learning models embedded in real-world decision-making systems, where the consequences of inaccurate predictions may be high, and better insight into model uncertainty can help avoid bad outcomes due to misplaced trust in model predictions. The paper focuses on training-free uncertainty estimation, in which uncertainty estimates are driven by input perturbations and modifications to ‘grey-box’ models (such as application of dropout to a trained model). These methods avoid the computational burden of applying uncertainty estimation during the training process. The paper evaluates the methods on two regression tasks: image super-resolution and monocular depth estimation. Evaluation metrics are based on the correlation between predictive variance and loss. The paper concludes that the training-free methods examined give comparable or better performance for uncertainty estimation, compared with state-of-the-art methods applied during training. The paper addresses an important topic, and is generally well-executed and well-written. However, it focuses on a narrow set regression tasks (with little indication that results generalize further), and uses evaluation metrics that are nonstandard and not well-substantiated. For these reasons, I recommend that the paper be rejected. A systematic study of training-free uncertainty estimation, which is the goal of the paper, would be a significant contribution. The paper chooses to focus on regression tasks, which is a valid narrowing of scope (although the implication that uncertainty estimation for classification is a solved problem, via entropy methods, I believe is incorrect). Image super-resolution and monocular depth estimation, the two regression tasks that the paper examines, are important but do not represent a broad enough set of tasks to justify claims about the general performance of the methods (though they enable the observations in equations (1) and (2)). The performance of the methods on these two tasks alone I don’t believe is a significant enough finding to warrant acceptance to ICLR. My second major concern with the paper is in the choice of evaluation metrics, which are based on the correlation between variance of predictions (over a sample of perturbations) and the value of the loss. While I understand how these metrics provide intuition on the quality of uncertainty estimates (i.e. when a model is “more wrong”, ideally it should be more uncertain), the paper does not give a good justification for the validity of the metrics beyond intuition (nor does the cited paper, Zhang et al. (2019)). The correlation metrics obscure a lot of information on the relationship between the predictive distribution and the true values, and without further justification, it is difficult to imagine trusting the resulting uncertainty estimates in a consequential downstream decision-making task. Proper scoring rules (Gneiting and Rafferty (2007), Lakshminarayanan et al. (2017)) could provide a more principled and useful basis for assessing the quality of uncertainty estimates. Smaller points: - The paper makes the implicit assumption that training/test sets are IID. While I understand that prediction on out-of-distribution (OOD) samples is beyond the paper’s scope, high uncertainty on OOD samples is a desirable property for uncertainty estimation methods, and some discussion of this or indication of the methods’ performance on OOD samples would strengthen the paper. - The paper implies that uncertainty estimation via input perturbation is a novel contribution; a similar method was used by Liang et al. (2018; https://arxiv.org/pdf/1706.02690.pdf) for the different but related task of out-of-distribution detection. - Is there a way to derive error bars for the results in Tables 1 and 2? Some of the correlation coefficients, reported to 4 decimal places, are very close and it’s unclear whether the results are significantly different.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Training-Free Uncertainty Estimation for Neural Networks
Uncertainty estimation is an essential step in the evaluation of the robustness for deep learning models in computer vision, especially when applied in risk-sensitive areas. However, most state-of-the-art deep learning models either fail to obtain uncertainty estimation or need significant modification (e.g., formulating a proper Bayesian treatment) to obtain it. None of the previous methods are able to take an arbitrary model off the shelf and generate uncertainty estimation without retraining or redesigning it. To address this gap, we perform the first systematic exploration into training-free uncertainty estimation. We propose three simple and scalable methods to analyze the variance of output from a trained network under tolerable perturbations: infer-transformation, infer-noise, and infer-dropout. They operate solely during inference, without the need to re-train, re-design, or fine-tune the model, as typically required by other state-of-the-art uncertainty estimation methods. Surprisingly, even without involving such perturbations in training, our methods produce comparable or even better uncertainty estimation when compared to other training-required state-of-the-art methods. Last but not least, we demonstrate that the uncertainty from our proposed methods can be used to improve the neural network training.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper addresses the problem of deriving better predictive uncertainty estimates from pre-trained models. As the paper notes in the introduction, this is an important problem for machine learning models embedded in real-world decision-making systems, where the consequences of inaccurate predictions may be high, and better insight into model uncertainty can help avoid bad outcomes due to misplaced trust in model predictions. The paper focuses on training-free uncertainty estimation, in which uncertainty estimates are driven by input perturbations and modifications to ‘grey-box’ models (such as application of dropout to a trained model). These methods avoid the computational burden of applying uncertainty estimation during the training process. The paper evaluates the methods on two regression tasks: image super-resolution and monocular depth estimation. Evaluation metrics are based on the correlation between predictive variance and loss. The paper concludes that the training-free methods examined give comparable or better performance for uncertainty estimation, compared with state-of-the-art methods applied during training. The paper addresses an important topic, and is generally well-executed and well-written. However, it focuses on a narrow set regression tasks (with little indication that results generalize further), and uses evaluation metrics that are nonstandard and not well-substantiated. For these reasons, I recommend that the paper be rejected. A systematic study of training-free uncertainty estimation, which is the goal of the paper, would be a significant contribution. The paper chooses to focus on regression tasks, which is a valid narrowing of scope (although the implication that uncertainty estimation for classification is a solved problem, via entropy methods, I believe is incorrect). Image super-resolution and monocular depth estimation, the two regression tasks that the paper examines, are important but do not represent a broad enough set of tasks to justify claims about the general performance of the methods (though they enable the observations in equations (1) and (2)). The performance of the methods on these two tasks alone I don’t believe is a significant enough finding to warrant acceptance to ICLR. My second major concern with the paper is in the choice of evaluation metrics, which are based on the correlation between variance of predictions (over a sample of perturbations) and the value of the loss. While I understand how these metrics provide intuition on the quality of uncertainty estimates (i.e. when a model is “more wrong”, ideally it should be more uncertain), the paper does not give a good justification for the validity of the metrics beyond intuition (nor does the cited paper, Zhang et al. (2019)). The correlation metrics obscure a lot of information on the relationship between the predictive distribution and the true values, and without further justification, it is difficult to imagine trusting the resulting uncertainty estimates in a consequential downstream decision-making task. Proper scoring rules (Gneiting and Rafferty (2007), Lakshminarayanan et al. (2017)) could provide a more principled and useful basis for assessing the quality of uncertainty estimates. Smaller points: - The paper makes the implicit assumption that training/test sets are IID. While I understand that prediction on out-of-distribution (OOD) samples is beyond the paper’s scope, high uncertainty on OOD samples is a desirable property for uncertainty estimation methods, and some discussion of this or indication of the methods’ performance on OOD samples would strengthen the paper. - The paper implies that uncertainty estimation via input perturbation is a novel contribution; a similar method was used by Liang et al. (2018; https://arxiv.org/pdf/1706.02690.pdf) for the different but related task of out-of-distribution detection. - Is there a way to derive error bars for the results in Tables 1 and 2? Some of the correlation coefficients, reported to 4 decimal places, are very close and it’s unclear whether the results are significantly different.
assistant
This paper addresses the problem of deriving better predictive uncertainty estimates from pre-trained models. As the paper notes in the introduction, this is an important problem for machine learning models embedded in real-world decision-making systems, where the consequences of inaccurate predictions may be high, and better insight into model uncertainty can help avoid bad outcomes due to misplaced trust in model predictions. The paper focuses on training-free uncertainty estimation, in which uncertainty estimates are driven by input perturbations and modifications to ‘grey-box’ models (such as application of dropout to a trained model). These methods avoid the computational burden of applying uncertainty estimation during the training process. The paper evaluates the methods on two regression tasks: image super-resolution and monocular depth estimation. Evaluation metrics are based on the correlation between predictive variance and loss. The paper concludes that the training-free methods examined give comparable or better performance for uncertainty estimation, compared with state-of-the-art methods applied during training. The paper addresses an important topic, and is generally well-executed and well-written. However, it focuses on a narrow set regression tasks (with little indication that results generalize further), and uses evaluation metrics that are nonstandard and not well-substantiated. For these reasons, I recommend that the paper be rejected. A systematic study of training-free uncertainty estimation, which is the goal of the paper, would be a significant contribution. The paper chooses to focus on regression tasks, which is a valid narrowing of scope (although the implication that uncertainty estimation for classification is a solved problem, via entropy methods, I believe is incorrect). Image super-resolution and monocular depth estimation, the two regression tasks that the paper examines, are important but do not represent a broad enough set of tasks to justify claims about the general performance of the methods (though they enable the observations in equations (1) and (2)). The performance of the methods on these two tasks alone I don’t believe is a significant enough finding to warrant acceptance to ICLR. My second major concern with the paper is in the choice of evaluation metrics, which are based on the correlation between variance of predictions (over a sample of perturbations) and the value of the loss. While I understand how these metrics provide intuition on the quality of uncertainty estimates (i.e. when a model is “more wrong”, ideally it should be more uncertain), the paper does not give a good justification for the validity of the metrics beyond intuition (nor does the cited paper, Zhang et al. (2019)). The correlation metrics obscure a lot of information on the relationship between the predictive distribution and the true values, and without further justification, it is difficult to imagine trusting the resulting uncertainty estimates in a consequential downstream decision-making task. Proper scoring rules (Gneiting and Rafferty (2007), Lakshminarayanan et al. (2017)) could provide a more principled and useful basis for assessing the quality of uncertainty estimates. Smaller points: - The paper makes the implicit assumption that training/test sets are IID. While I understand that prediction on out-of-distribution (OOD) samples is beyond the paper’s scope, high uncertainty on OOD samples is a desirable property for uncertainty estimation methods, and some discussion of this or indication of the methods’ performance on OOD samples would strengthen the paper. - The paper implies that uncertainty estimation via input perturbation is a novel contribution; a similar method was used by Liang et al. (2018; https://arxiv.org/pdf/1706.02690.pdf) for the different but related task of out-of-distribution detection. - Is there a way to derive error bars for the results in Tables 1 and 2? Some of the correlation coefficients, reported to 4 decimal places, are very close and it’s unclear whether the results are significantly different.