When presented with Out-of-Distribution (OOD) examples, deep neural networks yield confident, incorrect predictions. Detecting OOD examples is challenging, and the potential risks are high. In this paper, we propose to detect OOD examples by identifying inconsistencies between activity patterns and class predicted. We find that characterizing activity patterns by feature correlations and identifying anomalies in pairwise feature correlation values can yield high OOD detection rates for far-from-distribution examples. We identify anomalies in the pairwise feature correlations by simply comparing each pairwise correlation value with its respective range observed over the training data. Unlike many approaches, this can be used with any pre-trained softmax classifier and does not require access to OOD data for fine-tuning hyperparameters, nor does it require OOD access for inferring parameters. The method is applicable across a variety of architectures and vision datasets and, for the important and surprisingly hard task of detecting far-from-distribution outliers, it generally performs better than or equal to state-ofthe-art OOD detection methods (including those that do assume access to OOD examples).
This paper uses Gram matrices for OOD detection. This enables the reliable detection of far-from-distribution examples, which is a long-standing and surprisingly difficult problem in OOD detection.
This is the best paper in my batch due to the strength of the results. However, this paper should more accurately reflect the contribution: this helps with far-from-distribution examples, not near-distribution yet OOD examples.
For instance, I used their code and found that their technique leads to an AUROC of 79.01% when using CIFAR-10 as the in-distribution and CIFAR-100 as the OOD set. Likewise, with their code I found having CIFAR-100 as in and CIFAR-10 as out gives an AUROC of 67.95%. This is much worse than currently existing techniques. Hence the paper should re-frame or qualify their results as helping with far-from-distribution detection or the detection of obvious anomalies. This paper makes a solid stride in improving the detection of garbage inputs, but the paper should modify its message so as not to suggest this helps with all currently considered OOD detection tasks.
Small comments:
> Lee et al. (2018b)—to the best of our knowledge, the current SOTA technique by a significant margin
This should be again qualified and expanded. If you assume access to knowledge of the test distribution, then Mahalanobis is easily the best. If not, the Outlier Exposure is best. If you assume access to no extra data during training, then the maximum softmax probability + rotation prediction is best [1].
> However, while the OE method is able to generalize across different non-training distributions, it does not achieve the SOTA rates of Lee et al. (2018b) on most cases.
There are different senses of state-of-the-art and these should be qualified.
Does this technique do better if you do 5th and 9th percentile instead of min and max? Is it important to do the min and max with training examples instead of validation examples? (Not a pressing question.)
> Can work without access to OOD validation examples?
Table 1 is deceptive. OE does not need the "validation" examples.
I suggest two columns instead of one. "Can this work without knowledge of OOD test examples? Does this use OOD training examples?"
Show AUROC and AUPR. Detection accuracy is an unusual metric relative to AUPR (OOD as positive).
Show CIFAR-10 vs CIFAR-100 in the tables or I'll downgrade my rating, since otherwise the paper is not leaving an accurate impression.
Since OE is complementary, perhaps this technique can be combined to tackle these near-distribution cases?
The title is confusing. "Zero-Shot" could be applied to various techniques in this space. Perhaps emphasize Gram matrices?
In their code:
        validation_indices = random.sample(range(len(all_test_deviations)),int(0.1*len(all_test_deviations)))
        test_indices = sorted(list(set(range(len(all_test_deviations)))-set(validation_indices)))
These indices change with every power, which is unrealistic. Please fix the sets beforehand.
Since neural style transfer, which uses Gram matrices, works much better with VGG architectures than ResNets, does this technique work better with VGG architectures?
[1] Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty. Hendrycks et al. NeurIPS 2019.