Published as a conference paper at ICLR 2018 MITIGATING ADVERSARIAL EFFECTS THROUGH RAN- DOMIZATION
Convolutional neural networks have demonstrated high accuracy on various tasks in recent years. However, they are extremely vulnerable to adversarial examples. For example, imperceptible perturbations added to clean images can cause convolutional neural networks to fail. In this paper, we propose to utilize randomization at inference time to mitigate adversarial effects. Specifically, we use two randomization operations: random resizing, which resizes the input images to a random size, and random padding, which pads zeros around the input images in a random manner. Extensive experiments demonstrate that the proposed randomization method is very effective at defending against both single-step and iterative attacks. Our method provides the following advantages: 1) no additional training or fine-tuning, 2) very few additional computations, 3) compatible with other adversarial defense methods. By combining the proposed randomization method with an adversarially trained model, it achieves a normalized score of 0.924 (ranked No.2 among 107 defense teams) in the NIPS 2017 adversarial examples defense challenge, which is far better than using adversarial training alone with a normalized score of 0.773 (ranked No.56). The code is public available at https: //github.com/cihangxie/NIPS2017_adv_challenge_defense.
The authors propose a simple defense against adversarial attacks, which is to add randomization in the input of the CNNs. They experiment with different CNNs and published adversarial training techniques and show that randomized inputs mitigate adversarial attacks. 
Pros:
(+) The idea introduced is simple and flexible to be used for any CNN architecture
(+) Experiments on ImageNet1k prove demonstrate its effectiveness
Cons:
(-) Experiments are not thorougly explained
(-) Novelty is extremely limited
(-) Some baselines missing
The experimental section of the paper was rather confusing. The authors should explain the experiments and the settings in the table, as those are not very clear. In particular, it was not clear whether the defense model was trained with the input randomization layers? Also, in Tables 1-6, how was the target model trained? How do the training procedures of target vs. defense model differ? In those tables, what is the testing procedure for the target model and how does it compare to the defense model? 
The gap between the target and defense model in Table 4 (ensemble pattern attack scenario) shrinks for single step attack methods. This means that when the attacker is aware of the randomization parameters, the effect of randomization might diminish. A baseline that reports the performance when the attacker is fully aware of the randomization of the defender (parameters, patterns etc.) is missing but is very useful.
While the experiments show that the randomization layers mitigate the effect of randomization attacks, it's not clear whether the effectiveness of this very simple approach is heavily biased towards the published ways of generating adversarial attacks and the particular problem (i.e. classification). The form of attacks studied in the paper is that of additive noise. But there is many types of attacks that could be closely related to the randomization procedure of the input and that could lead to very different results.