Predictive Uncertainty Estimation via Prior Networks
Estimating how uncertain an AI system is in its predictions is important to improve the safety of such systems. Uncertainty in predictive can result from uncertainty in model parameters, irreducible data uncertainty and uncertainty due to distributional mismatch between the test and training data distributions. Different actions might be taken depending on the source of the uncertainty so it is important to be able to distinguish between them. Recently, baseline tasks and metrics have been defined and several practical methods to estimate uncertainty developed. These methods, however, attempt to model uncertainty due to distributional mismatch either implicitly through model uncertainty or as data uncertainty. This work proposes a new framework for modeling predictive uncertainty called Prior Networks (PNs) which explicitly models distributional uncertainty. PNs do this by parameterizing a prior distribution over predictive distributions. This work focuses on uncertainty for classification and evaluates PNs on the tasks of identifying out-of-distribution (OOD) samples and detecting misclassification on the MNIST and CIFAR-10 datasets, where they are found to outperform previous methods. Experiments on synthetic and MNIST and CIFAR-10 data show that unlike previous non-Bayesian methods PNs are able to distinguish between data and distributional uncertainty.
The authors develop prior networks, an approach to identifying misclassification on in and out of distribution examples. The work largely builds upon Malinin et al, 2017, adapting their method to the Dirichlet case. The experiments are simple but lack controls and may be too simple to validate the method. Overall: Quality: There do not appear to be any obvious errors. There is no model uncertainty in this paper as a dirac is used for the model prior, so why spend time talkimg about it? Clarity: The presentation at times seems convoluted and several confusing terms are introduced for apparently the same thing, even where standard terms exist for these in statistics. Some of the explanation is unclear and needs refinement. Originality: It is a fairly straightforward adaptation of Malinin et al, 2017 to the Dirichlet case (using a net for the concentration parameter instead of the variance). Significance: The experimental evaluation is weak. (See comments below) Comments: Intro: The epistemic/aleatoric uncertainty terms are not useful. It has been related to the explained/unexplained variance in the law of total variance, which differs from reducible/irreducible uncertainty in general. Model/data/distributional uncertainty are much clearer notions, are well defined, and are analogous to terms in statistical learning theory. It's worth noting that variantional methods have also been applied to (1), most recently see Graves, 2011, and Blundell et al, 2015, Kingma et al, 2015, Louizos and Welling, 2016. Eq (2)/(3): this explanation needs improving. 1) Instead of P(y=w_c), why not use capital letters for the random variables and lower case (with subscripts if you like) for their values? 2) eq (3) should include the conditional for each P(y = w_c) term: they are not marginals, it's just a vector of the terms in the sum of eq (2). Line 94 talks about decision boundaries, but there are not any defined in text or the figures (you need to specify a loss over choices to identify the boundary). I'm also not sure aim to have increasingly varied decision boundaries far from training data, how is this justified? Section 3: it seems sensible to mention that what is proposed is a hierarchical Bayesian model, in the sense there are now three layers: the likelihood, the per-data prior distribution and the global prior distribution. This idea has been extensively explored in non-neural bayesian models (e.g., LDA). (7) and (8) were a surprise. They effectively eliminate the whole hierarchy that was introduced in Section 3. It makes the whole story about Bayesian inference a bit overboard: in fact, there is very little to do with Bayes here. Indeed, it's not stated explicitly that I could see, but p(w_c|mu) = mu_c and so, fact, everything simplifies to eq (12) which shows that essentially the prediction is just a softmax over a neural network applied to the inputs in eq (13). Whereas Malinin et al, 2017 use (14) to fit Gaussians, this work use it to fit categorical data. That appears to be the main distinction to prior work. This work gives no further justification to this heuristic (they appear to set the coefficient trading off between these two tasks to one without justification). Certainly it looks like parameter estimation by this approach will be biased, but what other properties does it possess? The experiments consist of an experiment on a toy Gaussian discrimination task and on MNIST vs OMNIGLOT out of distribution discrimination. It's not unremarkable that PN worked on the toy task (but it is not really a strong scientific test; there's no baseline, maybe the task is too easy). The MNIST vs OMNIGLOT tasks is concerning. Performance of the baseline models is already high on these tasks, and no error bars are included, hence we do not know if the task is simply too easy and there is no difference in performance between the proposal and other obvious approaches.