Adversarial attacks were shown to be very effective in degrading the performance of neural networks. By slightly modifying the input, an almost identical input is misclassified by the network. To address this problem, we adopt the universal learning framework. In particular, we follow the recently suggested Predictive Normalized Maximum Likelihood (pNML) scheme for universal learning, whose goal is to optimally compete with a reference learner that knows the true label of the test sample but is restricted to use a learner from a given hypothesis class. In our case, the reference learner is using his knowledge on the true test label to perform minor refinements to the adversarial input. This reference learner achieves perfect results on any adversarial input. The proposed strategy is designed to be as close as possible to the reference learner in the worst-case scenario. Specifically, the defense essentially refines the test data according to the different hypotheses, where each hypothesis assumes a different label for the sample. Then by comparing the resulting hypotheses probabilities, we predict the label and detect whether the sample is adversarial or natural. Combining our method with adversarial training we create a robust scheme which can handle adversarial input along with detection of the attack. The resulting scheme is demonstrated empirically.
Summary:
This paper focuses on the area of adversarial defense -- both to improve robustness and to detect adversarially perturbed images. They approach the problem from the universal prediction / universal learning framework.
Modivation:
I had a hard time understanding the motivation of this work -- specifically the connection to the universal learning framework which to be fair I am unfamiliar with. In the absence of the universal learning framework formalism the method proposed here is quite simple and in my opinion clever -- create a new prediction based on performing an adversarial attack to each target class. What value does universal learning bring to this?
Second, I do not follow the intuition for the chosen hypothesis class -- why work off of refined images in the first place? Is there some reason to believe this will improve robustness?
Finally, the view that adversarial defense and attack are important topics to explore is under some debate. I am not considering this as part of my review but I would encourage the authors to look at [1].
Writing:
The writing was clear and typo free.
Experiments:
Overall the experiments seemed inconclusive.
Section 5 shows robustness against the unmodified / unrefined model (the attacks are done on the base model not the refined model). Given that these attacks are performed against the unmodified model then evaluated on the modified model the results seem a bit unfair / harder to interpret. The authors note this, and in Section 6 explore the "Adaptive Adversary" setting.
The results presented are performed on Mnist and Cifar10. Overall the results were not convincing to me. Table 1 shows mixed performance -- a drop in natural accuracy in all cases, decreases in FGSM. The main increase in performance is in the PGD. This was noted, but understanding in more depth why this method helps here will hopefully lead to improved performance in FGSM as well.
Figure 2a shows very weak correlations. Figure 2b seems promising but also not necessarily a surprise given that the adversarial examples are generated against the base model and not the refined model.
For section 6, one risk is that the BPDA attack doesn't successfully work. Having some more proof that the attacks presented here are strong would greatly improve the work.
Larger scale experiments would of course be nice and strengthen the paper but more importantly it would be great to see some form of toy example or demonstration of the principle improving robustness as well over just results. Something to probe the mechanism of action for example.
Finally, having some comparisons to other defense strategies would improve this paper.
Rating:
Given the gap between the universal learning framework and the method proposed, as well as the inconclusive experiments at this point I would not recommend the paper for acceptance.
[1] https://arxiv.org/abs/1807.06732