Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts. They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution. In this work, we investigate robust classification with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises. Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs. Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classifiers. Experimentally, we find that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10. We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective. Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classification.
Update: I thank the authors' for their response, and have read the other reviews.
This paper demonstrates some theoretical and practical limitations on the use of likelihood based generative models for detecting adversarial examples. They construct a simple counterexample showing that there are adversarial examples for an arbitrarily accurate model (as measured by KL) that are not detectable by diminished likelihood of the model (as the dimension increases). Extending the work of Gilmer et al, this proves that there can be no general robustness guarantee for conditional generative models (Bayes classifiers). They provide compelling empirical evidence that while conditional normalizing flows trained on MNIST can be effective in detecting and defending adversarial attacks, these models trained on CIFAR10 are not. Surprisingly, it is shown that linear interpolations between images of different classes yield higher likelihoods for the CIFAR10 models and that class has little impact on model likelihoods. This goes some way in explaining why the detection is not effective on CIFAR, but questions still remain.
The paper makes fairly modest claims, but does a good job at demonstrating them and shedding some light on the issue. The experiments are thorough and fit into a growing body of evidence that the likelihoods of normalizing flows and other image based likelihood models may not be that informative or well calibrated, where past work has focused on out-of-distribution detection. My only major complaint with the paper is that it is not clear to what extent the theoretical and practical problems are related. As mentioned in the paper, the counterexample construction depends on the geometry of the data rather than the learning model. It could be that for both the MNIST and CIFAR10 datasets, the geometry is such that robustness garauntees are possible, and that the discrepancy in detection and interpolation arises because the normalizing flow has modeled the MNIST distribution much better than the CIFAR10 distribution. In this case we might hope that using conditional likelihood models for adversarial detection can be made effective, but that effort needs to be placed into improving the modeling capability. It's not obvious how to probe this distinction, but it would be good if this was given some thought in the paper. Also it would be good to see the attack detection numbers on BG-MNIST.
Comments:
Difficulty in training conditional generative models:
I believe in the two papers you cite the models do not use the label as input, but rather there is a separate model for each class? The overfitting is likely why the models had slightly lower conditional likelihood. As an aside, there are a couple of other examples of conditional normalizing flow models on images that use a mixture of Gaussians in the latent space [1], [2].
eq. 4: In the paper it is said that the second term in eq 4 is at most log(C), because the uniform distribution would have this value and that therefore this is negligibly small in comparison to the other term. Why exactly is this the case, couldn't the data entropy term be smaller in principle even if it's larger in practice? Or is the argument that the data entropy term scales with the dimensionality, but the label term does not leading to an imbalance? This could use some clarification.
A3: What is meant by ‘While the ground-truth likelihoods for the padded and un-padded datapoints are the same due to independence of the uniform noise and unit density of the noise’ in the appendix section A3? Wouldn't the ground truth negative log likelihoods would increase by the entropy of the uniform noise? Also, then in the bits per dimension calculations is the dimension the number of unpadded dimensions or the padded dimensions?
[1] Izmailov, Pavel, et al. "Semi-Supervised Learning with Normalizing Flows." Workshop on Invertible Neural Nets and Normalizing Flows, International Conference on Machine Learning. 2019.
[2] Atanov, Andrei, et al. "Semi-Conditional Normalizing Flows for Semi-Supervised Learning." Workshop on Invertible Neural Nets and Normalizing Flows, International Conference on Machine Learning. 2019.