To facilitate a wide-spread acceptance of AI systems guiding decision making in real-world applications, trustworthiness of deployed models is key. That is, it is crucial for predictive models to be uncertainty-aware and yield well-calibrated (and thus trustworthy) predictions for both in-domain samples as well as under domain shift. Recent efforts to account for predictive uncertainty include post-processing steps for trained neural networks, Bayesian neural networks as well as alternative non-Bayesian approaches such as ensemble approaches and evidential deep learning. Here, we propose an efficient yet general modelling approach for obtaining well-calibrated, trustworthy probabilities for samples obtained after a domain shift. We introduce a new training strategy combining an entropy-encouraging loss term with an adversarial calibration loss term and demonstrate that this results in well-calibrated and technically trustworthy predictions for a wide range of perturbations. We comprehensively evaluate previously proposed approaches on different data modalities, a large range of data sets including sequence data, network architectures and perturbation strategies and observe that our modelling approach substantially outperforms existing state-of-the-art approaches, yielding well-calibrated predictions under domain drift.
The paper presents a method for calibrating neural networks on in- and out-of-distribution data using two additional loss terms: entropy-encouraging loss term which maximizes softmax probabilities for wrong classes and adversarial calibration loss term which pushes confidences to match accuracy on adversarial examples. The idea of using adversarial calibration training is interesting and promising, however, the clarity of the paper needs significant improvement and there are several issues which need to be addressed; for this reason, I recommend a weak reject for the current version.
1. One of the contributions listed in the paper is that authors “illustrate the limitations of entropy as measure for trustworthy predictions and introduce a new metric to quantify technical trustworthiness based on the concept of calibration”; the section 2.1 discusses this in detail. The proposed metric is expected calibration error (ECE) averaged over different levels of noise perturbations applied to data during test time. This metric assumes that for a given dataset we know in advance what kind of noise perturbations can be implied to cause out-of-domain/domain-shift scenarios during test time. This assumption most likely doesn’t hold in real-world applications since domain shifts may be of various kinds. Moreover, calibration may not make sense at all for out-of-domain data if test inputs don’t belong to any of the train classes (as opposed to entropy of predictive distribution which still can be computed and is expected to be higher for such inputs). The metric is also dependent on the considered noise level range; the plot of ECE vs noise level can be illustrative and informative while the averaged value of ECE over all noise levels can be misleading (at least, we may want to have some discount factor to account for the fact that with noise level 0 we strongly care about calibration, while for very high noise levels calibration doesn’t give us much information since the useful patterns in data may be corrupted and it may be impossible even for humans to classify such objects, e.g. on Figure 1-top-left for noise level 100, the digit 6 is corrupted so much that we wouldn’t care about calibration for such inputs but just want to maximize the entropy).
The paper also claims “Recent efforts in terms of evaluating predictive uncertainty have focused on entropy as measure for uncertainty-awareness for predictions under domain shift.” In previous work addressing uncertainty in domain shift [1], not only entropy but other various metrics are considered for out-of-domain detection (Brier score, thresholded confidences, etc), these metrics don’t depend on particular perturbations and are very informative.
The introduced metric, ECE for different noise labels, makes sense only in the toy scenarios where we can control the noise level, however, it would still be better to either look at the plots or to use some discounted averaged ECE over different noise levels, but not equal average. So the significance of this contribution (introduction of new metric) is limited.  
2. The clarity of the paper could be significantly improved.
(a) Figure 1: the top left plot suggests that for high noise level “wrong predictions are often made with high confidence” (so the entropy of this distribution is low) while the top right plot shows that on average predictive entropy gradually grows and is high for high noise levels, so the top right plot is probably not a representative example and it might be misleading to claim this is an “often” case.
(b) Could you please clarify what you mean by “after removing non-misleading evidence” in section 2.2.1? In the next sentence, the remaining probability is probably distribution uniformly across C-1, not C, classes. The predictive entropy loss term essentially maximizes the probabilities of wrong classes with a lower coefficient. How would you support the claim that the loss surface is unchanged?
The loss term is “parameter-free” but still we need to tune a coefficient lambda_S for it.
(c) The adversarial loss equation in section 2.2.2 is written as L2-norm of a scalar value, why is L2-norm needed? The acc(B_m) is not differentiable so it is probably just considered as constant in the paper? Please comment on that. 
(d) Section 3.2 and Figure 4. On Figure 4 Middle and Right plots have different colors than those listed on legend, please, fix this. On the middle plot, both FALCON and EDL have high ECE for noise levels <50 which indicates that they are both highly underconfident for in-distribution data and low noise levels, while other methods have close to 0 ECE on low noise levels. The authors only comment on EDL underconfidence: “it is worth noting that EDL has a substantially higher ECE for in-domain predictions, reflecting under-confident predictions on the test set”, and not on FALCON underconfidence. However, using the proposed score, ECE averaged over all noise levels (Table 2), it may look like the method is doing a good job, while underconfidence problem is revealed when looking at Figure 4. This is also an illustration of the concern about the proposed metric I raised in point 1.
3.  It would help to have an ablation study in the main text of the paper showing the significance of each loss term: in the appendix Figure S1, it is shown that adding adversarial loss to standard and entropy loss helps. Is the entropy loss needed at all? How does performance change if we only have standard and adversarial loss? How sensitive is performance to the choice of lambda_avd and lambda_s?
[1] Ovadia, Yaniv, et al. "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift." arXiv preprint arXiv:1906.02530 (2019).