BEYOND PIXEL NORM-BALLS: PARAMETRIC ADVERSARIES USING AN ANALYTICALLY DIFFERENTIABLE RENDERER
Many machine learning image classifiers are vulnerable to adversarial attacks, inputs with perturbations designed to intentionally trigger misclassification. Current adversarial methods directly alter pixel colors and evaluate against pixel norm-balls: pixel perturbations smaller than a specified magnitude, according to a measurement norm. This evaluation, however, has limited practical utility since perturbations in the pixel space do not correspond to underlying real-world phenomena of image formation that lead to them and has no security motivation attached. Pixels in natural images are measurements of light that has interacted with the geometry of a physical scene. As such, we propose a novel evaluation measure, parametric normballs, by directly perturbing physical parameters that underly image formation. One enabling contribution we present is a physically-based differentiable renderer that allows us to propagate pixel gradients to the parametric space of lighting and geometry. Our approach enables physically-based adversarial attacks, and our differentiable renderer leverages models from the interactive rendering literature to balance the performance and accuracy trade-offs necessary for a memory-efficient and scalable adversarial data augmentation workflow.
Summary:
This work presents a method to generate adversary examples capable of fooling a neural network classifier. Szegedy et al. (2013) were the first to expose the weakness of neural networks against adversarial attacks, by adding a human-imperceptible noise to images to induce misclassification. Since then, several works tackled this problem by modifying the image directly in the pixel space: the norm-balls convention. The authors argue that this leads to non-realistic attacks and that a network would not benefit from training with these adversarial images when performing in the real world. Their solution and contributions are parametric norm-balls: unlike state-of-the-art methods, they perform perturbations in the image formation space, namely the geometry and the lighting, which are indeed perturbations that could happen in real life. For that, they defined a differentiable renderer by making some assumptions to simplify its expression compared to solving a light transport equation. The main simplifications are the direct illumination to gain computation efficiency and the distant illumination and diffuse material assumptions to represent lighting in terms of spherical harmonics as in Ramamoorthi et al. (2001), which require only 9 parameters to approximate lighting. This allows them to analytically derivate their loss function according to the geometry and lighting and therefore generate their adversary examples via gradient descent. They show that their adversary images generalize to other classifiers than the one used (ResNet). They then show that injecting these images into the training set increase the robustness of WideResNet against real attacks. These real attack images were taken by the authors in a laboratory with varying illumination.
Strength:
- The proposed perturbations in the image formation space simulate the real life scenario attacks.
- The presented results show that the generated adversary images do fool the classifier (used to compute the loss) but also new classifiers (different than the one used to compute the loss). As a consequence the generated adversary images increase the robustness of the considered classifier. 
- Flexibility in their cost function allows for diverse types of attacks: the same modified geometry can fool a classifier in several views, either into detecting the same object or detecting different false objects under different views. 
Major comments:
- Method can only compute synthetic adversary examples, unlike state-of-the-art.
- The main contribution claimed by the author is that their perturbations are realistic and that it would help better increase the robustness of classifiers against real attacks. However, they do not give any comparison to the state-of-the-art methods as is expected. 
Minor comments:
- Even if the paper is well written, they are still some typos.