Recent works find that DNNs are vulnerable to adversarial examples, whose changes from the benign ones are imperceptible and yet lead DNNs to make wrong predictions. One can find various adversarial examples for the same input to a DNN using different attack methods. In other words, there is a population of adversarial examples, instead of only one, for any input to a DNN. By explicitly modeling this adversarial population with a Gaussian distribution, we propose a new black-box attack called N ATTACK. The adversarial attack is hence formalized as an optimization problem, which searches the mean of the Gaussian under the guidance of increasing the target DNN's prediction error. N ATTACK achieves 100% attack success rate on six out of eleven recently published defense methods (and greater than 90% for four), all using the same algorithm. Such results are on par with or better than powerful state-of-the-art white-box attacks. While the white-box attacks are often model-specific or defense-specific, the proposed black-box N ATTACK is universally applicable to different defenses.
In this work the authors use a score-based adversarial attack (based on the natural evolution strategy (NES)) to successfully attack a multitude of defended networks, with success rates rivalling the best gradient-based attacks.
As confirmed by the authors in a detailed and very open response to a question of mine, the attack introduced here is actually equivalent to [1]. While the attack itself is not novel (which will require a major revision of the manuscript), the authors point out the following contributions over [1]:
* Attack experiments here go way beyond Ilyas et al. in terms of Lp metrics, different defense models, different datasets and transferability.
* Different motivation/derivation of NES.
* Concept of adversarial distributions.
* Regression network for good initialization.
* Introduction of accuracy-iterations plots.
My main concerns are as follows:
* The review of the prior literature, in particular on score-based and decision-based defences (the latter of which are not even mentioned), is very limited and is framed wrongly. In particular, the statement “However, existing black-box attacks are weaker than their white-box counterparts” is simply not true: as an example, the most prominent decision-based attack [2] rivals white-box attacks on vanilla DNNs as well as defended networks [3].
* The concept of adversarial distributions is not new but is common in the literature of real-world adversarials that are robust to transformations and perturbations (like gaussian noise), check for example [4]. In [4] the concept of _Expectation Over Transformation (EOT)_ is introduced, which is basically the generalised concept of the expectation over gaussian perturbations introduced in this work.
* While I like the idea of accuracy-iterations plots, the idea is not new, see e.g. the accuracy-iterations plot in [2] (sample-based, Figure 6), the loss-iterations plot in [5] or the accuracy-distortion plots in [3]. However, I agree that these type of visualisation or metric is not as widespread as it should be.
Hence, in summary the main contribution of the paper is the application of NES against different defence models, datasets and Lp metrics as well as the use of a regression network for initialisation. Along this second point it would be great if the authors would be able to demonstrate substantial gains in the accuracy-query metric. In any case, in the light of previous literature a major revision of the manuscript will be necessary.
[1] Ilyas et al. (2018) “Black-box Adversarial Attacks with Limited Queries and Information” (https://arxiv.org/abs/1804.08598) 
[2] Brendel et al. (2018) “Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models” (https://arxiv.org/abs/1712.04248)
[3] Schott et al. (2018) “Towards the first adversarially robust neural network model on MNIST” (https://arxiv.org/abs/1805.09190)
[4] Athalye et al. (2017) “Synthesizing Robust Adversarial Examples” (https://arxiv.org/pdf/1707.07397.pdf)
[5] Madry et al. {2017) “Towards Deep Learning Models Resistant to Adversarial Attacks” (https://arxiv.org/pdf/1706.06083.pdf)