Inhibited Softmax for Uncertainty Estimation in Neural Networks
We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains.
 # Summary The paper proposes a new method for computing output uncertainty estimates in DNNs for classification problems. The authors claim that replacing the softmax output layer for an Inhibit Softmax layer (a modified softmax with an additional constant additive term in the denominator, proposed in Saito et al., 2016) matches the state-of-the-art methods for uncertainty estimation an outperforms them in out-of-distribution detection tasks. While the original idea is simple, the final method is convoluted, including several modifications (section 3.1.) to deal with an ill-behaved optimization objective, making it difficult to evaluate why the method works in the end. Finally, the experimental section needs to be extended. The proposed metrics are presented as the gold standard to measure uncertainty calibration; however, it is known that this is itself an open problem due to the fact that ground truth labels about uncertainty are not available. The paper should be extended with experiments that use a simulated data to gain a further understanding of what kind of uncertainty it is capture by the model, as well as considering additional metrics like the brier score, histograms of predictive entropy, …. Also, several state-of-the-art inference methods for BNNs that improve the results of the original backprop paper are missing in the experimental section. # Details The key idea is to use an Inhibited softmax output layer (instead the classical softmax) to get a better estimation of the output uncertainty (epistemic, aleatoric and distributional). They claim that the method matches other state-of-the-art methods in the field and outperform them in out-of-distribution detection tasks. However, the reason why this may be true is unclear and the experimental section needs to be strengthen. Regarding the inhibited softmax layer, they propose to decompose the log-likelihood in two term: the log-likelihood of the classical softmax plus a term modeling a certainty score that is maximized. What the method is doing is adding a term to all the probabilities which effectively is a simple smoothing of the output probabilities, leaving out some probability for when the test data is far from the training one. This value is fix as a hyper-parameter. In particular “c” dictates the probability of the instance belonging to a new class, and so, fixing it ad-hoc means that you determine how far a test data should be from the training set in order to consider it out-of-distribution. It seems then a bad idea to fix this with independence of the dataset/model we are looking at. The paper would benefit from a deeper analysis of the method and connections with other methods in the literature, e.g. connections to the open world classification problem would be of interest. Authors should elaborate on Section 3.1.. This section summarized 4 modifications that seems to be needed in order for the proposed method to work. Each of these modifications are explained in just a short paragraph, and it seems that choosing correctly these values is critical for the final performance of the algorithm. The number of hyper-parameters to tune increases making more difficult to understand why the method works (despite the analysis in the appendix). In the experimental section, I would encourage the authors to propose some experiments using synthetic data where they have more control about the uncertainty of the generated data. These experiments would allow to visualize several properties of the uncertainty estimator and offer more insights to the reader. I would also like to see how the current method performs in the low data regime: when you have a small training set and an over parameterized DNN. In this case, I would expect that the method overfit since all the uncertainty of the lower levels is collapsed before reaching the output. Finally, in recent years, several advances in variational inference have been proposed that improve upon the basic version of Backprop (Blundell et al., 2015), e.g., https://arxiv.org/pdf/1703.01961.pdf, https://arxiv.org/pdf/1511.06233.pdf … that should be included in the experimental section.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Inhibited Softmax for Uncertainty Estimation in Neural Networks
We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 # Summary The paper proposes a new method for computing output uncertainty estimates in DNNs for classification problems. The authors claim that replacing the softmax output layer for an Inhibit Softmax layer (a modified softmax with an additional constant additive term in the denominator, proposed in Saito et al., 2016) matches the state-of-the-art methods for uncertainty estimation an outperforms them in out-of-distribution detection tasks. While the original idea is simple, the final method is convoluted, including several modifications (section 3.1.) to deal with an ill-behaved optimization objective, making it difficult to evaluate why the method works in the end. Finally, the experimental section needs to be extended. The proposed metrics are presented as the gold standard to measure uncertainty calibration; however, it is known that this is itself an open problem due to the fact that ground truth labels about uncertainty are not available. The paper should be extended with experiments that use a simulated data to gain a further understanding of what kind of uncertainty it is capture by the model, as well as considering additional metrics like the brier score, histograms of predictive entropy, …. Also, several state-of-the-art inference methods for BNNs that improve the results of the original backprop paper are missing in the experimental section. # Details The key idea is to use an Inhibited softmax output layer (instead the classical softmax) to get a better estimation of the output uncertainty (epistemic, aleatoric and distributional). They claim that the method matches other state-of-the-art methods in the field and outperform them in out-of-distribution detection tasks. However, the reason why this may be true is unclear and the experimental section needs to be strengthen. Regarding the inhibited softmax layer, they propose to decompose the log-likelihood in two term: the log-likelihood of the classical softmax plus a term modeling a certainty score that is maximized. What the method is doing is adding a term to all the probabilities which effectively is a simple smoothing of the output probabilities, leaving out some probability for when the test data is far from the training one. This value is fix as a hyper-parameter. In particular “c” dictates the probability of the instance belonging to a new class, and so, fixing it ad-hoc means that you determine how far a test data should be from the training set in order to consider it out-of-distribution. It seems then a bad idea to fix this with independence of the dataset/model we are looking at. The paper would benefit from a deeper analysis of the method and connections with other methods in the literature, e.g. connections to the open world classification problem would be of interest. Authors should elaborate on Section 3.1.. This section summarized 4 modifications that seems to be needed in order for the proposed method to work. Each of these modifications are explained in just a short paragraph, and it seems that choosing correctly these values is critical for the final performance of the algorithm. The number of hyper-parameters to tune increases making more difficult to understand why the method works (despite the analysis in the appendix). In the experimental section, I would encourage the authors to propose some experiments using synthetic data where they have more control about the uncertainty of the generated data. These experiments would allow to visualize several properties of the uncertainty estimator and offer more insights to the reader. I would also like to see how the current method performs in the low data regime: when you have a small training set and an over parameterized DNN. In this case, I would expect that the method overfit since all the uncertainty of the lower levels is collapsed before reaching the output. Finally, in recent years, several advances in variational inference have been proposed that improve upon the basic version of Backprop (Blundell et al., 2015), e.g., https://arxiv.org/pdf/1703.01961.pdf, https://arxiv.org/pdf/1511.06233.pdf … that should be included in the experimental section.
assistant
 # Summary The paper proposes a new method for computing output uncertainty estimates in DNNs for classification problems. The authors claim that replacing the softmax output layer for an Inhibit Softmax layer (a modified softmax with an additional constant additive term in the denominator, proposed in Saito et al., 2016) matches the state-of-the-art methods for uncertainty estimation an outperforms them in out-of-distribution detection tasks. While the original idea is simple, the final method is convoluted, including several modifications (section 3.1.) to deal with an ill-behaved optimization objective, making it difficult to evaluate why the method works in the end. Finally, the experimental section needs to be extended. The proposed metrics are presented as the gold standard to measure uncertainty calibration; however, it is known that this is itself an open problem due to the fact that ground truth labels about uncertainty are not available. The paper should be extended with experiments that use a simulated data to gain a further understanding of what kind of uncertainty it is capture by the model, as well as considering additional metrics like the brier score, histograms of predictive entropy, …. Also, several state-of-the-art inference methods for BNNs that improve the results of the original backprop paper are missing in the experimental section. # Details The key idea is to use an Inhibited softmax output layer (instead the classical softmax) to get a better estimation of the output uncertainty (epistemic, aleatoric and distributional). They claim that the method matches other state-of-the-art methods in the field and outperform them in out-of-distribution detection tasks. However, the reason why this may be true is unclear and the experimental section needs to be strengthen. Regarding the inhibited softmax layer, they propose to decompose the log-likelihood in two term: the log-likelihood of the classical softmax plus a term modeling a certainty score that is maximized. What the method is doing is adding a term to all the probabilities which effectively is a simple smoothing of the output probabilities, leaving out some probability for when the test data is far from the training one. This value is fix as a hyper-parameter. In particular “c” dictates the probability of the instance belonging to a new class, and so, fixing it ad-hoc means that you determine how far a test data should be from the training set in order to consider it out-of-distribution. It seems then a bad idea to fix this with independence of the dataset/model we are looking at. The paper would benefit from a deeper analysis of the method and connections with other methods in the literature, e.g. connections to the open world classification problem would be of interest. Authors should elaborate on Section 3.1.. This section summarized 4 modifications that seems to be needed in order for the proposed method to work. Each of these modifications are explained in just a short paragraph, and it seems that choosing correctly these values is critical for the final performance of the algorithm. The number of hyper-parameters to tune increases making more difficult to understand why the method works (despite the analysis in the appendix). In the experimental section, I would encourage the authors to propose some experiments using synthetic data where they have more control about the uncertainty of the generated data. These experiments would allow to visualize several properties of the uncertainty estimator and offer more insights to the reader. I would also like to see how the current method performs in the low data regime: when you have a small training set and an over parameterized DNN. In this case, I would expect that the method overfit since all the uncertainty of the lower levels is collapsed before reaching the output. Finally, in recent years, several advances in variational inference have been proposed that improve upon the basic version of Backprop (Blundell et al., 2015), e.g., https://arxiv.org/pdf/1703.01961.pdf, https://arxiv.org/pdf/1511.06233.pdf … that should be included in the experimental section.