Published as a conference paper at ICLR 2018 COUNTERING ADVERSARIAL IMAGES USING INPUT TRANSFORMATIONS
This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier. Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images. The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methods.
To increase robustness to adversarial attacks, the paper fundamentally proposes to transform an input image before feeding it to a convolutional network classifier. The purpose of the transformation is to erase the high-frequency signals potentially embedded by an adversarial attack.
Strong points:
* To my knowledge, the proposed defense strategy is novel (even if the idea of transformation has been introduced at https://arxiv.org/abs/1612.01401). 
* The writing is reasonably clear (up to the terminology issues discussed among the weak points), and introduces properly the adversarial attacks considered in the work.
* The proposed approach really helps in a black-box scenario (Figure 4). As explained below, the presented investigation is however insufficient to assess whether the proposed defense helps in a true white-box scenario. 
Weak points:
* The black-box versus white-box terminology is not appropriate, and confusing. In general, black-box means that the adversary ignores everything from the decision process. Hence, in this case, the adversary does not know about the classification model, nor the defensive method, when used. This corresponds to Figure 3. On the contrary, white-box means that the adversary knows everything about the classification method, including the transformation implemented to make it more robust to attacks. Assimilating the parameters of the transform to a secret key is not correct because those parameters could be inferred by presenting many image samples to the transform and looking at the outcome of the transformation (which is supposed to be available in a 'white-box' paradigm) for those samples. 
* Using block diagrams would definitely help in presenting the training/testing and attack/defense schemes investigated in Figure 3, 4, and 5.
* The paper does not discuss the impact of the denfense strategy on the classification performance in absence of adversity.
* The paper lacks of positioning with respect to recent related works, e.g. 'Adversary Resistant Deep Neural Networks with an Application to Malware Detection' in KDD 2017, or 'Building Adversary-Resistant Deep Neural Networks without
Security through Obscurity' at https://arxiv.org/abs/1612.01401. 
* In a white-box scenario, the adversary knows about the transformation and the classification model. Hence, an effective and realistic attack should exploit this knowledge. Designing an attack in case of a non differentiable transformation is obviously not trivial since back-propagation can not be used. However, since the proposed transformation primarily aim at removing the high frequency pattern induced by the attack, one could for example design an attack that account for a (linear and differentiable) low-pass filter transformation. Another example of attack that account for transformation knowledge (and would hopefully be more robust than the attacks considered in the manuscript) could be one that alternates between a conventional attack and the transformation.
* If I understand correctly, the classification model considered in Figure 3 has been trained on original images, while the one in Figure 4 has been trained on transformed images. However, in absence of attack, they both achieve 76% accuracy. Is it correct? Does it mean that the transformation does not affect the classification accuracy at all?
Overall, the works investigates an interesting idea, but lacks maturity to be accepted. Therefore, I would only recommend acceptation if room.
Minor issues:
Typo on p7: to change*s*
Clarify poor formulations:
* p1: 'enforce model-specific strategies that enforce model properties such as invariance and smoothness via the learning algorithm or regularization schemes'. 
* p1: 'too simple to remove adversarial perturbations from input images sufficiently'