Owing to the susceptibility of deep learning systems to adversarial attacks, there has been a great deal of work in developing (both empirically and certifiably) robust classifiers, but the vast majority has defended against single types of attacks. Recent work has looked at defending against multiple attacks, specifically on the MNIST dataset, yet this approach used a relatively complex architecture, claiming that standard adversarial training can not apply because it "overfits" to a particular norm. In this work, we show that it is indeed possible to adversarially train a robust model against a union of norm-bounded attacks, by using a natural generalization of the standard PGD-based procedure for adversarial training to multiple threat models. With this approach, we are able to train standard architectures which are robust against ∞ , 2 , and 1 attacks, outperforming past approaches on the MNIST dataset and providing the first CIFAR10 network trained to be simultaneously robust against ( ∞ , 2 , 1 ) threat models, which achieves adversarial accuracy of 46.1% against the union of ( ∞ , 2 , 1 ) perturbations with radius = (0.03, 0.5, 12).
This paper adversarially trains models against l_p norms where p is of there different values. They then propose a method which does somewhat better than the obvious way of adversarially training against more than one l_p perturbation.
The motivation for the paper is limited, in that they suggest previous works have suggested adversarial training itself "overfits" to the given l_p norm. This isn't surprising that it works, since the straightforward baseline works. They make it seem surprising by suggesting that ABS suggested adversarial training is doomed and cannot provide robustness to l_1, l_2, l_\infty norms simultaneously. The other motivation is that this is a step toward studying an expanded threat model, but the authors have not demonstrated that the learned representations are any bit more robust to common corruptions (could the authors show the generalization performance on CIFAR-10-C or generalization to unforeseen corruptions?). Without further evidence, we are left to believe this only helps for this narrow threat model. Overall the paper is deficient in creativity and generality, so I vote for rejection.
Small comments:
> take more time than a single norm,  it is a step closer towards the end goal of truly robust models, with adversarial robustness against all perturbations.
Please show model performance on CIFAR-10-C since if the model is more robust, it should hopefully be more robust to stochastic adversaries.
> has claimed that adversarial training “overfits” to the particular type of perturbation used to generate the adversarial examples
Wouldn't this be that l_\infty training fits specifically to l_\infty examples, not that robust optimization cannot handle more than one norm at a time? Who is claiming that?
> First, we show that even simple aggregations of different adversarial attacks can achieve competitive universal robustness against multiple perturbations models without resorting to complex architectures.
I am not sure this was in doubt. The phrase "universal robustness" is misleading.
How were the budgets chosen for l_2 and l_1? Those values seem small.