summaries: The authors present a simple algorithm based on the statistics of neural activations of deep networks to detect out-of-distribution samples. The idea is to use the existing running estimate of mean and variance within BatchNorm layers to construct feature representations that are later fed into a simple linear classifier. The authors demonstrate superior performance over the previous state-of-art in the standard evaluation setting and provide fascinating insights and empirical analysis of their method. There has been recent interest in using statistics and information summary measures to evaluate what deep nets are trying to do. Following the line of work, the paper suggests to use mean and variance of Z-scores accumulated across all layers/channels as features to distinguish ID and OOD samples. Simple idea but needs some work in its current format. 
abstractText: The ability to detect when an input sample was not drawn from the training distribution is an important desirable property of deep neural networks. In this paper, we show that a simple ensembling of first and second order deep feature statistics can be exploited to effectively differentiate in-distribution and out-of-distribution samples. Specifically, we observe that the mean and standard deviation within feature maps differs greatly between in-distribution and out-of-distribution samples. Based on this observation, we propose a simple and efficient plug-and-play detection procedure that does not require re-training, pre-processing or changes to the model. The proposed method outperforms the state-of-the-art by a large margin in all standard benchmarking tasks, while being much simpler to implement and execute. Notably, our method improves the true negative rate from 39.6% to 95.3% when 95% of in-distribution (CIFAR-100) are correctly detected using a DenseNet and the out-of-distribution dataset is TinyImageNet resize. The source code of our method will be made publicly available.
references.authors: N. Akhtar A.S. Mian
references.title: Threat of adversarial attacks on deep learning in computer vision: A survey
references.venue: IEEE Access,
references.authors: D. Amodei C. Olah J. Steinhardt P. Christiano J. Schulman D. Mané
references.title: Concrete problems in AI safety, June 2016
references.authors: D. Bahdanau K. Cho Y. Bengio
references.title: Neural machine translation by jointly learning to align and translate
references.authors: C.M. Bishop
references.title: Pattern recognition and machine learning
references.venue: Information Science and Statistics. Springer-Verlag,
references.authors: J. Deng W. Dong R. Socher L.-J. Li K. Li L. Fei-Fei
references.title: ImageNet: A large-scale hierarchical image database
references.venue: In Computer Vision and Pattern Recognition, Florida,
references.authors: I. Goodfellow J. Pouget-Abadie M. Mirza B. Xu D. Warde-Farley S. Ozair A. Courville Y. Bengio
references.title: Generative adversarial nets
references.venue: Advances in Neural Information Processing Systems,
references.authors: G. Goswami N.K. Ratha A. Agarwal R. Singh M. Vatsa
references.title: Unravelling robustness of deep learning based face recognition against adversarial attacks
references.venue: In AAAI Conference on Artificial Intelligence,
references.authors: C. Guo G. Pleiss Y. Sun K.Q. Weinberger
references.title: On calibration of modern neural networks
references.venue: In International Conference on Machine Learning,
references.authors: K.J. Han A. Chandrashekaran J. Kim I. Lane
references.title: The CAPIO 2017 conversational speech recognition system, December 2018
references.authors: K. He X. Zhang S. Ren J. Sun
references.title: Deep residual learning for image recognition
references.venue: arXiv preprint arXiv:1512.03385
references.authors: K. He X. Zhang S. Ren J. Sun
references.title: Identity mappings in deep residual networks
references.venue: In European Conference on Computer Vision,
references.authors: D. Hendrycks K. Gimpel
references.title: A baseline for detecting misclassified and out-of-distribution examples in neural networks
references.venue: In International Conference on Learning Representations,
references.authors: D. Hendrycks M. Mazeika T. Dietterich
references.title: Deep anomaly detection with outlier exposure, 2018
references.authors: G.E. Hinton L. Deng D. Yu G.E. Dahl A. Mohamed N. Jaitly A. Senior V. Vanhoucke P. Nguyen T.N. Sainath B. Kingsbury
references.title: Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
references.venue: IEEE Signal Processing Magazine,
references.authors: G. Huang Z. Liu L. van der Maaten K.Q. Weinberger
references.title: Densely connected convolutional networks
references.venue: In Computer Vision and Pattern Recognition, Hawaii,
references.authors: S. Ioffe C. Szegedy
references.title: Batch normalization: Accelerating deep network training by reducing internal covariate shift
references.venue: In International Conference on Machine Learning,
references.authors: A. Krizhevsky
references.title: Learning multiple layers of features from tiny images
references.venue: Technical report,
references.authors: A. Krizhevsky I. Sutskever G.E. Hinton
references.title: ImageNet classification with deep convolutional neural networks
references.venue: In Advances in Neural Information Processing Systems,
references.authors: G.E. Hinton L. van der Maaten
references.title: Visualizing high-dimensional data using t-SNE
references.venue: Journal of Machine Learning Research,
references.authors: K. Lee H. Lee J. Shin
references.title: Training confidence-calibrated classifiers for detecting outof-distribution samples
references.venue: In International Conference on Learning Representations,
references.authors: K. Lee H. Lee J. Shin
references.title: A simple unified framework for detecting out-of-distribution samples and adversarial attacks, July 2018b
references.authors: S. Liang Y. Li R. Srikant
references.title: Enhancing the reliability of out-of-distribution image detection in neural networks
references.venue: In International Conference on Learning Representations,
references.authors: Larry M. Manevitz Malik Yousef
references.title: One-class SVMs for document classification
references.venue: Journal of Machine Learning Research,
references.authors: Y. Netzer Y. Wang A. Coates A. Bissacco B. Wu A.Y. Ng
references.title: Reading digits in natural images with unsupervised feature learning
references.venue: In Workshop on Deep Learning and Unsupervised Feature Learning,
references.authors: S. Ruder
references.title: An overview of gradient descent optimization algorithms, September 2016
references.authors: M. Sabokrou M. Fayyaz M. Fathy Z. Moayed R. Klette
references.title: Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes, September 2016
references.authors: J. Su D. Vasconcellos Vargas S. Kouichi
references.title: One pixel attack for fooling deep neural networks, October 2017
references.authors: A. van den Oord Y. Li I. Babuschkin K. Simonyan O. Vinyals K. Kavukcuoglu G. van den Driessche E. Lockhart L.C.C. Rus F. Stimberg N. Casagrande D. Grewe S. Noury S. Dieleman E. Elsen N. Kalchbrenner H. Zen A. Graves H. King T. Walters D. Belov D. Hassabis
references.title: Parallel WaveNet: Fast high-fidelity speech synthesis
references.venue: Technical report, Google Deepmind,
references.authors: A. Vyas N. Jammalamadaka D. Das X. Zhu B. Kaul T.L. Willke
references.title: Out-of-distribution detection using an ensemble of self supervised leave-out classiers
references.venue: In European Conference on Computer Vision,
references.authors: P. Xu K.A. Ehinger Y. Zhang A. Finkelstein S.R. Kulkarni J. Xiao
references.title: TurkerGaze: Crowdsourcing saliency with webcam based eye
references.authors: F. Yu Y. Zhang S. Song A. Seff J. Xiao
references.title: LSUN: Construction of a large-scale image dataset using deep learning with humans
references.venue: in the loop,
references.authors: S. Zagoruyko N. Komodakis
references.title: Wide residual networks
references.venue: In British Machine Vision Conference,
sections.heading: 1 INTRODUCTION
sections.text: In the past few years, deep neural networks (DNNs) (Goodfellow et al., 2016) have settled as the state-of-the art-techniques in many difficult tasks in a plurality of domains, such as image classification (Krizhevsky et al., 2012), speech recognition (Hinton et al., 2012; Han et al., 2018), and machine translation (Bahdanau et al., 2014; van den Oord et al., 2017). This recent progress has been mainly due to their high accuracy and good generalization ability when dealing with realworld data. Unfortunately, DNNs are also highly confident when tested against unseen samples, even if the samples are vastly different from the ones employed during training (Hendrycks & Gimpel, 2017). Moreover, several works have shown that such deep networks are easily fooled by minor perturbations to the input (Goodfellow et al., 2014; Su et al., 2017). Obtaining a calibrated confidence score from a deep neural network is a problem under continuous investigation (Hendrycks & Gimpel, 2017) and a major thread in artificial intelligence (AI) safety (Amodei et al., 2016). In fact, knowing when the model is wrong or inaccurate has a direct impact in many production systems, such as self-driving cars, authentication and disease identification (Akhtar & Mian, 2018; Goswami et al., 2018), to name a few.
  Guo et al. (2017) showed that despite producing significantly low classification errors, DNNs confidence scores are not faithful estimates of the true certainty. Their experiments confirmed that depth, width, weight decay, and batch normalization are the main reasons for overconfident scores. Moreover, they demonstrated that a simple and yet powerful method of temperature scaling in the softmax scores is an effective way to improve calibrate a DNNs confidence. While calibrating the classifier’s output to represent a faithful likelihood from the training (in-distribution) data has effective solutions, the problem of detecting whether or not the samples are generated from a known distribution (out-of-distribution), is still an open problem (Hendrycks & Gimpel, 2017).
  One straightforward approach to calibrate the classifier’s confidence in order to detect samples whose distribution differs from the training samples distribution is to train a secondary classifier that digests both in-distribution (ID) and out-of-distribution (OOD) data so that anomalies are scored differently from ID samples, as performed in Hendrycks et al. (2018). Re-training a network, however, can be computationally intensive and may even be intractable, since the number of OOD samples is virtually infinite. Other solutions rely on training both classification and generative neural net-
  works for OOD using a multi-task loss (Lee et al., 2018a), or using unsupervised fully convolutional networks as done by Sabokrou et al. (2016) to detect OOD in video samples. All these methods, however, have a major drawback: they require re-training a modified model using a different loss function possibly with additional parameters, which increases the computational burden, and demands access to the entire original (and probably huge) training data.
  In this work, we propose a novel OOD sample detection method that explores low-level statistics from feature layers. The statistics are obtained directly from the batch normalization layers (Ioffe & Szegedy, 2015), requiring no extra computations during training time, no changes to the network architecture and loss functions, nor preprocessing of the input image. During test time, the proposed method extracts statistics from selected layers and combines them into an OOD-ness score via a linear classifier. Throughout this paper, we observe that the mean and standard deviation of a given channel in a layer differ greatly between ID and OOD samples, which naturally motivates their use as features to be employed by an OOD detector. By selecting the BN layers of a network, we are able to normalize the features according to the learned batch statistics. The effectiveness of the proposed method is validated in two state-of-the-art DNN architectures (DenseNet and ResNet) (Huang et al., 2017; He et al., 2015; 2016; Zagoruyko & Komodakis, 2016) that are trained for image classification tasks in popular datasets. The proposed approach achieves state-of-the-art performance, surpassing all competitors by a large margin in all tested scenarios, while being much more efficient. Notably, our method only requires one forward pass while Liang et al. (2018); Lee et al. (2018b); Vyas et al. (2018) require two forward and one backward passes.
  The rest of the paper is organized as follows. Section 2 describes prior work on OOD samples detection. Section 3 introduces the proposed method, whereas Section 4 details all experiments and compares the results with state-of-the-art methods. Finally, we draw our conclusions in Section 5.
sections.heading: 2 PREVIOUS WORK
sections.text: In this section, we describe recent prior work on OOD detection methods. A summary of all methods described can be seen in Table 1.
  Hendrycks & Gimpel (2017) proposed a baseline method based on the posterior distribution (i.e. softmax scores). They showed that well-trained models tend to produce higher scores for ID samples than for OOD ones. Hence their method comprises of applying a threshold on the softmaxnormalized output of a classifier. If the largest score is below the threshold, then the sample is considered OOD.
  Liang et al. (2018) continued the aforementioned line of work and proposed the Out-of-Distribution detector for Neural networks (ODIN), which includes a temperature scaling T ∈ R+∗ to the softmax classifier as in Guo et al. (2017). The authors in ODIN argued that a good manipulation of T eases the separation between in- and out-of-distribution samples. Allied to that, they also incorporated small perturbations to the input (inspired by Goodfellow et al. (2014)) whose goal is to increase the softmax score of the predicted class. Liang et al. (2018) found that this kind of perturbation has a stronger effect on ID samples than on OOD ones, increasing the separation between ID and OOD samples. ODIN outperforms the baseline method (Hendrycks & Gimpel, 2017) by a fair
  margin; however, it is three times slower due to the two forward and one backward passes needed to preprocess the input, while (Hendrycks & Gimpel, 2017) only requires one forward pass.
  Vyas et al. (2018) describes a novel loss function, called margin entropy loss, over the softmax output that attempts to increase the distance between ID and OOD samples. During training, they partition the training data itself into ID and OOD by assigning samples labeled as certain classes as OOD and use the different partitions to train an ensemble of classifiers that are then used to detect OOD samples during test time. They also use the input pre-processing step proposed in Liang et al. (2018), including temperature scaling.
  Lee et al. (2018b) is the most recent work on OOD detection that we have knowledge of. They show that the posterior distribution defined by a generative classifier (under Gaussian discriminant analysis) is equivalent to that of the softmax classifier, and the generative classifier eases the separation between in- and out-of-distribution samples. The confidence score is defined using the Mahalanobis distance between the sample and the closest class-conditional Gaussian distribution. They argue that abnormal samples can be better characterized in the DNN feature space rather than the output space of softmax-based posterior distribution as done in previous work (e.g., ODIN). Samples are pre-processed similarly as done in ODIN, but the confidence score is increased instead of the softmax one. To further improve the performance, they also consider intermediate generative classifiers for all layers in the network. The final OOD sample detector is computed as an ensemble of confidence scores, chosen by training a logistic regression on validation samples. This method also shows remarkable results for detection of adversarial attacks and for incremental learning.
sections.heading: 3 PROPOSED SOLUTION
sections.text: An OOD detector should incorporate information from the training data in a natural manner, without being directly influenced by the loss function, which is intrinsically related to the task which could be well-defined for ID samples but be meaningless for most OOD samples. Moreover, if the OOD method is more dependent on the training distribution, it should be able to be applied to a wide variety networks, and not be designed specifically for a given architecture.
  Our method is based on a very simple observation. Input samples with different distributions generate statistically distinct feature spaces in a DNN. In other words, the deep features of an ID sample are distinct from those of an OOD one. Moreover, when using a normalization scheme, such as BN, the features are normalized by the statistics of the ID batch during training, possibly leading to feature statistics that are more similar to the batch statistics, as depicted in Figure 2.
  The main problem then becomes how to summarize the feature space distribution for ID samples in a way that best discriminates between ID and OOD samples. In this work we show that using the first and second order statistics within each feature map performs remarkably well for this task.
  Figure 1 shows the proposed OOD detector, highlighting the feature statistics that are extracted for each channel at each BN layer. These statistics are then combined through a learned linear decision function, providing a measure of OOD-ness. The way we compute the per-channel statistics
  is formalized in Section 3.1. Finally, the linear classifier used to combine the statistics from the different layers is described in Section 3.2.
sections.heading: 3.1 LOW-ORDER FEATURE STATISTICS
sections.text: In the previous section, we motivated that characterizing the feature-space distributions might lead to a robust OOD detector. As a first approach, one could model these distributions using a nonparametric method to estimate the distribution of the features for each channel, which requires the computation of sufficient statistics using the training data or using a parametric method to fit the distribution (Bishop, 2007), which are both computationally intensive.
  Here, we propose to use only the mean and standard deviation computed along the spatial dimension for each channel to summarize the per-channel distribution. As it will be shown, these two statistics are sufficient to distinguish between ID and OOD. Moreover, because the mean and standard deviation of each channel are normalized by the running mean and variance computed during training by the BN layers (Ioffe & Szegedy, 2015), these statistics can be naturally combined within each layer to produce effective features for OOD detection. We describe such a procedure in what follows.
  Given the l-th BN layer with input tensor X ∈ RC×W×H , the output BN (l)c,i,j at channel c is
  Z (l)c,i,j = X (l)c,i,j − µ (l) c√
  [σ (l) c ]2 +
  , BN (l)c,i,j = Z (l) c,i,jγ (l) c + β (l) c , (1)
  where γ(l)c > 0 and β (l) c are the per channel per layer learned scaling and shifting parameters, > 0 is a small constant value for numerical stability, µ(l)c ∈ R and [σ(l)c ]2 ∈ R+ are the mean and variance estimated through a moving average using the batch statistics during training, and Z(l) ∈ RC×W×H is the normalized feature tensor. It is worth noting that the statistics are calculated independently for each channel c at each layer l.
  In this paper, we conjecture that low-order statistics computed from either X (l)c,i,j or Z (l) c,i,j can be used to discriminate between ID and OOD samples. Given the unnormalized input X (l)c,i,j , we can
  compute the empirical mean m(l)c and standard deviation σ (l) c features for each channel c as
  m(l)c = 1
  WH ∑ i,j X (l)c,i,j , [s (l) c ] 2 = 1 WH ∑ i,j [X (l)c,i,j ] 2 − [m(l)c ]2, (2)
  and the features for the normalized tensor Z (l)c,i,j are defined as
  m̂(l)c = 1
  WH ∑ i,j Z (l)c,i,j = m (l) c − µc(l) σ (l) c
  [ŝ(l)c ] 2 =
  1
  WH ∑ i,j [Z (l)c,i,j ] 2 − [m̂(l)c ]2 =
  [ ŝ (l) c
  σ (l) c
  ]2 ,
  (3)
  i.e., the normalized mean feature m̂(l)c represents the difference between the sample mean m (l) c and the BN running mean µ(l)c weighted by the running standard deviation σ (l) c , whereas ŝ (l) c is the normalized standard deviation by the BN running standard deviation σ(l)c .
sections.heading: 3.2 AGGREGATION OF FEATURE STATISTICS
sections.text: Intra-layers aggregation. The features derived from low-order statistics (equation 3) can be readily used to train a predictor for ID/OOD discrimination. Of course, if they were produced for every feature map in the network, this would result in a feature vector of very high dimension, typically tens of thousands. Instead, we propose to combine these features within each BN layer, so that in the end we obtain one pair of features per layer: average mean and average variance. Taking advantage of the fact that features are approximately normalized by BN’s running statistics, we propose to simply average them for all channels within a layer. Thus, each layer yields the following features, for the normalized case:
  m̄(l) = 1
  Cl ∑ c m̂(l)c , [s̄ (l)]2 = 1 Cl ∑ c [ŝ(l)c ] 2, (4)
  where Cl is the number of channels in layer l. Note that doing this aggregation amounts to computing the mean and standard deviation of all normalized features at the given layer. Using averages of the low-order statistics could lead to issues in deeper layers, where activations are in general concentrated over fewer number of channels. In this case, the mean of the statistics over channels might not be an appropriate data reduction function. Nevertheless, as we show in the experiments section, this did not impact the performance of the proposed method, but more investigation is warranted.
  Inter-layers ensemble and final classification. Using all the features in equation 4, i.e., f = (m̄(1), s̄(1), ..., m̄(L), s̄(L)) we fit a simple logistic regression model h(f ;θ) with parameters θ ∈ R2L+1. The parameters of the linear model are learned using a separate validation set formed with ID samples (positive examples) and OOD samples (negative examples).
sections.heading: 4 EXPERIMENTS
sections.text: In this section, we evaluate the effectiveness of the proposed method in state-of-the-art deep neural architectures, such as DenseNet (Huang et al., 2017) and Wide ResNet (Zagoruyko & Komodakis, 2016), on several computer vision benchmark datasets: CIFAR (Krizhevsky, 2009), TinyImageNet, a subset of ImageNet (Deng et al., 2009), LSUN (Yu et al., 2015), and iSUN (Xu et al., 2015). We also use Gaussian noise and uniform noise as synthetic datasets. This evaluation protocol is the de facto standard in recent OOD detection literature (Hendrycks & Gimpel, 2017; Liang et al., 2018; Vyas et al., 2018; Lee et al., 2018b). All experiments were performed on four models trained from scratch (each one initialized with a different random seed) for each architecture, to account for variance in the model parameters. The code to reproduce all results is publicly available1.
  1Code for review avilable at https://www.dropbox.com/s/o652ufpznatvx8u/ ood-codes-review.zip?dl=0.
sections.heading: 4.1 SETUP
sections.text: Datasets: For backbone training, we use CIFAR-10 and CIFAR-100 datasets which have 10 and 100 classes respectively, both containing 50,000 images in the training set and 10,000 images in the test set. At test time, the test images from CIFAR-10 (CIFAR-100) are considered as ID (positive) samples. For OOD (negative) datasets, we test with datasets containing natural images, such as TinyImageNet resize and crop, LSUN resize and crop, and iSUN, as well as synthetic datasets, such as Gaussian/uniform noise, which is the same dataset setup as in Liang et al. (2018); Hendrycks & Gimpel (2017). This is summarized in Table 2. For all datasets, we did the validation/test set split following the procedure in Liang et al. (2018). Just for reproducibility, 1000 samples from the test set are separated in a validation set used for fitting the logistic regressor and hyper-parameter tuning. The remaining samples (unseen for both backbone model and OOD detector) are used for testing.
  Backbone training: Following Liang et al. (2018), we adopt the DenseNet (Huang et al., 2017) and Wide ResNet (Zagoruyko & Komodakis, 2016) architectures as our benchmark networks. For DenseNet, we use depth L = 100, growth rate k = 12, and zero dropout rate (DenseNet-BC-10012). For Wide ResNet, we also follow Liang et al. (2018), with L = 28 and widen factor of 10 (WRN-28-10). All hyperparameters are identical to their original papers. All networks were trained using stochastic gradient descent with Nesterov momentum (Ruder, 2016) and an initial learning rate of 0.1. We train the DenseNet-BC-100-12 for 300 epochs, with batch size 64, momentum 0.9, weight decay of 10−4 and decay the learning rate by a factor of 10 after epochs 150 and 225. We train the WRN-28-10 for 200 epochs, with batch size 128, momentum 0.9, weight decay 0.0005, and decay the learning rate by a factor of 10 after epochs 60, 120, and 160. Table 3 shows each network error rate over 4 independent runs each one initialized with a different random seed.
  Logistic regression: The logistic regressor is trained considering only the validation partitions for ID (positive examples) and OOD (negative examples) datasets (see Table 2). Using both mean and standard deviation as input (from equation 4), we have 50 features for WRN-28-10 models, and 198 features for DenseNet-BC-100-12 models. The training was performed using 5-fold cross-validation with the `2 minimization and the regularization factor being chosen as the best one (according to the 5-folds) among 10 values linearly spaced in the range 10−4 and 104.
  Evaluation metrics: To evaluate the proposed method, we use the following metrics:
  1. True negative rate (TNR) at 95% true positive rate (TPR). Let TP, TN, FP, and FN be the true positive, true negative, false positive, and false negative, respectively. The TNR is defined as TN/(TN+FP) whereas TPR is defined as TP/(TP+FN).
  2. Area under the receiver operating characteristic curve (AUROC). AUROC is the area under the FPR=1-TNR against TPR curve.
sections.heading: 4.2 MANIFOLD VISUALIZATION
sections.text: We applied t-SNE (L. van der Maaten, 2008) to visualize our high-dimensional feature space in order to see the similarities between ID/OOD samples. For this, we used one of the WRN-28-10 models trained with CIFAR-10 as ID dataset. We fitted the t-SNE using the ID and all OOD validation samples together using both mean and standard deviation features, and the result is shown in Figure 3 using a perplexity of 30. It is clear from the visualization that the proposed features are concentrated around well-defined clusters. Both synthetic OOD datasets have clear distinct behavior from the natural images ones, and it is straightforward to differentiate them. TinyImageNet (c), LSUN (c) are similar and have some intersection with TinyImageNet (r), LSUN (r). Interestingly, the clustering seems to reflect the dataset generation method (resizing or cropping). Most importantly, one can see that the OOD samples are in different clusters than the ID (CIFAR-10) ones, which indicates that this feature choice is adequate for separating them.
sections.heading: 4.3 PAIRWISE FITTING
sections.text: In this section, following other methods in the literature, we adjust the linear classifier for each ID/OOD pair. That is, for each pair of ID/OOD datasets, a different OOD classifier is trained using their respective validation samples.
  We performed a comparison between the proposed method and four recent methods described in Section 2 (Hendrycks & Gimpel, 2017; Lee et al., 2018b; Liang et al., 2018; Vyas et al., 2018). Since Lee et al. (2018b) did not test using Wide ResNet models and the same datasets as in Liang et al. (2018); Vyas et al. (2018), here we only show the intersection between them: DenseNet BC 100- 12 model, using CIFAR-10 (CIFAR-100) as in-distribution and TinyImageNet (resize) and LSUN (resize) as OOD distribution2. Extended results can be found in the appendix. For both Hendrycks & Gimpel (2017) and Liang et al. (2018) results, we reimplemented the method using their reference implementation3,4. For ODIN (Liang et al., 2018), we employed the same procedure as described by the authors to tune the methods parameters. A detailed description of the procedure can be found in Appendix B.1. For Lee et al. (2018b) and Vyas et al. (2018), we use the values presented on their papers (Table 2 and Table 3, respectively).
  The results are compiled in Table 4. Notably, our method outperforms the baseline and ODIN methods by a large margin, and yields better results than Lee et al. (2018b) and Vyas et al. (2018) in all tested cases without requiring any preprocessing, or changing the backbone model. In fact, when setting the OOD-ness threshold to obtain 95% TPR, our method is able to correctly detect all OOD samples from the test partition.
  2At the time of submission, Lee et al. (2018b) was still unpublished work 3Baseline: github.com/hendrycks/error-detection 4ODIN: github.com/ShiyuLiang/odin-pytorch
sections.heading: 4.4 TESTING ON UNSEEN OOD DATASETS
sections.text: We argue that the pairwise fitting scenario presented in the previous section is a limited performance measurement. In fact, many practical applications have OOD samples that do not come from a single distribution mode, and it might be infeasible to collect data from the many different modes of the OOD distribution (in general, some modes are unknown during training). A good OOD detector should be able to correctly identify samples from OOD distibutions for which its parameters have not been adjusted to. With this in mind, we propose a different, harder task in which an OOD detector is fitted to one, or a few, OOD datasets and then it is tested on all OOD datasets available. We note that this is not a standard practice in previous works, like in Liang et al. (2018). We begin by evaluating the generalization ability of our detector in some preliminary experiments, which motivate our decisions in the choice of OOD datasets for fitting the model and feature selection.
  Selecting Features: We evaluate the individual impact of each of the proposed features (i.e., layers average mean and standard deviation) by comparing the performance of the classifier with different features as inputs. In this experiment, we fit the linear classifier to a specific OOD validation dataset and test on all of the OOD test datasets available. Our performance metric is the TNR @ 95% TPR averaged over all the tested datasets. Table 5 presents the performance of the classifier for the WRN-28-10, averaged of the four available models, with CIFAR-100 as ID dataset. The classifier fitted using only standard deviation features still achieved very good performance generalizing very well to unknown OOD datasets. Since we are interested in designing an OOD detector that is able to differentiate ID samples from any OOD sample, all our results from now on are presented using the averaged standard deviations per layer as the only features used to train the linear decision function.
  Selecting OOD Dataset: To understand to what extent a classifier trained considering one OOD dataset is able to generalize and detect samples from other OOD datasets, we trained the logistic regression considering as positive examples 1000 samples from the ID dataset (CIFAR-100) and as negative examples 1000 samples of a given OOD dataset, using the WRN-28-10 backbone model. As motivated in the previous section, we use the averages of normalized standard deviation features (equation 4). The obtained logistic regressor was then evaluated on the remaining OOD test datasets (unseen by both backbone training and logistic regressor). This procedure was then repeated for each possible OOD dataset, and the results are summarized in Table 6. We see that all classifier fitted using only natural images are capable to generalize well over all other OOD sets, while this is not entirely true when fitting on random noise datasets. Also, fitting to all OODs validation sets (penultimate row), we can achieve even higher TNR scores over all test sets.
  Using no OOD Dataset: To further evaluate the effectiveness of the method, we also tested the extreme case where no OOD samples are available for training. To do this, we used an unsupervised
  algorithm (one-class SVM (Manevitz & Yousef, 2002) with RBF kernel), and we only fitted to ID samples (i.e., no OOD samples are seen in the training step). The unsupervised results are summarized in the last row of Table 6. As one can see, even the unsupervised method shows reasonable performance; showing again that in the proposed feature space the ID/OOD samples have different behavior. This corroborates the assumption that these features are a good indicator of OOD-ness.
  Comparison with ODIN (Liang et al., 2018): We compare the generalization capabilities of our method with the state-of-the-art technique ODIN (Liang et al., 2018), in this new harder task. We fit both OOD classifier to maximize their detection performance on TinyImageNet (c) and Gaussian validation sets (i.e., 2000 OOD samples), and evaluate on all OOD test datasets. For our model, we use standard deviation features as inputs. For ODIN, we tune its hyperparameters using the grid search described in B.1. The results, presented in Table 7, show that our method outperforms ODIN by a large margin, indicating better generalization to samples from unseen OOD datasets.
  Effect of feature normalization: We evaluated how much it helps to use the batch statistics computed by BN. As shown in Table 8, normalizing the latent space using the BN statistics before computing the features has clear advantages.
  Reduced validation data: We study if our method can correctly detect OOD samples even when a small number of samples is available. Figure 4 shows the TNR @ 95% TPR for WRN 28-10 trained on CIFAR-10 (CIFAR-100), where only 27, 45, 75, 150, 300, 700, 1.5k, 3k (all) test images (ID + TinyImageNet crop + Gaussian, equally divided) are used to fit 25 coefficients of our logistic regressor. Using our method, only 27 images (from each ID and OOD), are necessary to achieve an average of 87.6% of TNR @ 95% TPR.
sections.heading: 5 CONCLUSION
sections.text: Deep neural networks trained to maximize classification performance in a given dataset are extremely adapted to said dataset. The statistics of activations throughout the network for samples from the training distribution (in-distribution) are remarkably stable. However, when a sample from a different distribution (out-of-distribution) is given to the network, its activation statistics depart greatly from those of in-distribution samples. Based on this observation, we propose a very simple yet efficient method to detect out-of-distribution samples. Our method is based on computing averages of low-order statistics at the batch normalization layers of the network, and then use them as features in a linear classifier. This procedure is much simpler and efficient than current stateof-the-art methods, and outperforms them by a large margin in the traditional ID/OOD fitting task (as proposed in previous works). We evaluated all methods in the challenging task of fitting on a single OOD dataset and testing on samples from other (unseen) datasets. In this harder scenario, our method generalizes well to unseen OOD datasets, outperforming ODIN by an even larger margin. Moreover, we show some preliminary results that even in the extreme case where no OOD samples are used for the training (unsupervised) we get reasonable performance.
sections.heading: A SUPPLEMENTARY RESULTS FOR SECTION 3
sections.heading: B SUPPLEMENTARY RESULTS FOR SECTION 4
sections.heading: B.1 PARAMETER FINE-TUNING FOR ODIN
sections.text: The temperature T and noise magnitude hyperparameters were tuned using 1000 samples from the CIFAR test set as validation and evaluated on the remaining 9000 samples. The hyper-parameter
  tuning is carried out on each pair of in- and out-of-distribution samples, which is the same procedure presented in Liang et al. (2018) and Lee et al. (2018b). For reproducibility, a grid search is employed considering T ∈ {1, 1000} and with 21 linearly spaced values between 0 and 0.004 plus [0.00005, 0.0005, 0.0011].
sections.heading: B.2 ADDITIONAL BACKBONE MODEL
sections.text: We also train a Wide ResNet withL = 40 and widen factor of 4 (WRN-40-4) using the same training setup in Section 4, and the results for both training on CIFAR-10 and CIFAR-100 is depicted in Table 9.
sections.heading: B.3 OTHER EVALUATED METRICS
sections.text: To compare all methods described in Section 2 to our proposed method, we also use the following additional metrics:
  1. Area under precision-recall curve (AUPR). The precision is evaluated as TP/(TP+FP) and recall in this case is the TPR. The AUPR-in (AUPR-out) is defined when in-(out-of)distribution samples are considered as the positive ones.
sections.heading: B.4 SELECTING OOD DATASET USING SVHN AS ID
sections.text: We also tested using the Street View House Number (SVHN) dataset (Netzer et al., 2011) as ID and DenseNet BC 100-12 as backbone model. The pre-trained model is from Lee et al. (2018b), which has a test error rate of 3.58%. The results are displayed in Table 10.
  B.5 INTER-LAYER ENSEMBLE.
  In this experiment, we evaluate the effect of ensembling across the BN layers, as described in Section 3.2, using a logistic regression. As shown in Figure 7, aggregating confidence scores of intermediate layers indeed improves the detection performance of OOD samples. The left-most bin indicates the TNR at 95% of TPR using only the last BN layer. As we go to the right, we aggregate the shallower BN layers, and the right-most bin indicates that all the BN layers are being used. Another important insight is that deeper layers (on the left of Figure 7) contribute more for the separation between in- and out-of-distribution samples.
sections.heading: B.6 OOD DETECTION COMPARISON
sections.text: We also extended our Table 7 with the extra backbone model (WRN 40-4) and additional metrics. The results are shown in Table 11.
sections.heading: B.7 OOD DETECTION AT DIFFERENT TPR LEVELS
sections.text: We also tested our proposed model on different TPR levels, and the results are depicted in Table 12.
title: DETECTING OUT-OF-DISTRIBUTION SAMPLES USING LOW-ORDER DEEP FEATURE STATISTICS