summaries: This work proposes and studies two techniques to generate unadversarial images (images which cause a computer vision deep learning model to more reliably give accurate predictions). One technique is for creating unadversarial patches which can be overlaid on testing images to increase the robustness of a pre-trained classifier, the other is for creating unadversarial textures for 3D meshes which when applied to 3D meshes. The work provides strong empirical evidence for unadversarial patches on ImageNet and CIFAR, including results where images are corrupted after applying the patches. Evidence is provided for unadversarial textures on simulated data and a small scale real world experiment. In this work, the authors leverage techniques from the adversarial examples literature to design patches and textures that aid in classifying an object *correctly.* In a fixed-model setting, they construct patches / textures using gradient-based methods and demonstrate effectiveness on clean CIFAR-10 and ImageNet and various robustness benchmarks such as ImageNet-C (common perturbations). The authors also compare their patches / textures to simple baselines such as QR codes, smaller-sized images from the training set, and predefined fixed patterns. This work introduces and studies "unadversarial" input modifications (patches or object textures) designed to *increase* classifier accuracy, particularly in corrupted images. The work is inspired by research on adversarial examples (in particular "adversarial patches" which trick a classifier into predicting a particular class when the patch is present in the scene, regardless of what other class might be present).  Given access to a known, fixed, pre-trained network, a small patch (or the entire object texture) are optimized by gradient descent to increase prediction accuracy. When applied, the method is shown to be quite effective for increasing classification accuracy (and performance on a simple regression task) under a variety of simulated image corruptions (blur, fog, etc.). In addition, the patch method is shown to significantly aid classification even when printed and affixed to real physical objects. This paper exploits the  known-sensitivity of deep models to perturbation in the input data in order to improve (rather than decrease) the performance of a pre-trained model in a classification task. Specifically, the authors propose to alter the input data (i.e. images)  by 1) adding a patch to the image 2) altering the texture of specific objects. The paper shows experimentally that this improves both in-domain performance and robustness to data corruptions. The paper considers a scenario where a fixed computer vision system (such as CNN based classification or regression models) is given and one tries to adapt the design of objects in order to increase their chance of being correctly detected by the system. This setting is similar to the design of marker systems for computer vision with the difference that the detection algorithm is considered fixed.  Two approaches are presented to design (i) patches that can be printed and sticked onto objects and (ii) complete textures that cover the surface of 3D objects. Both approaches follow the design of adversarial perturbations with the difference that instead of maximizing the loss of the system, it is now minimized to improve its performance instead of weakening it.  Experiments evaluate how these approaches can improve the overall performance of the given system as well as its robustness against a variety of different corruptions. Experiments are performed on synthetic 2D and 3D data, as well as on real objects, where the designed patches are printed out and attached to physical objects. Overall, they demonstrate consistent and significant improvements over non-modified objects and simple baselines.
abstractText: We study a class of computer vision settings wherein one can modify the design of the objects being recognized. We develop a framework that leverages this capability—and deep networks’ unusual sensitivity to input perturbations—to design “robust objects,” i.e., objects that are explicitly optimized to be confidently classified. Our framework yields improved performance on standard benchmarks, a simulated robotics environment, and physical-world experiments.1
authors.name: Hadi Salman
authors.name: Andrew Ilyas
authors.name: Logan Engstrom
authors.name: Ashish Kapoor
references.authors: Syed Muhammad Abbas Salman Aslam Karsten Berns Abubakr Muhammad
references.title: Analysis and Improvements in AprilTag Based State Estimation
references.authors: Anish Athalye Logan Engstrom Andrew Ilyas Kevin Kwok
references.title: Synthesizing Robust Adversarial Examples
references.venue: In: International Conference on Machine Learning (ICML)
references.authors: Michael A Alcorn Qi Li Zhitao Gong Chengfei Wang Long Mai Wei-Shinn Ku Anh Nguyen
references.title: Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects
references.venue: In: Conference on Computer Vision and Pattern Recognition (CVPR)
references.authors: Anurag Arnab Ondrej Miksik Philip HS Torr
references.title: On the robustness of semantic segmentation models to adversarial attacks
references.venue: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
references.authors: Battista Biggio Igino Corona Davide Maiorca Blaine Nelson Nedim Šrndić Pavel Laskov Giorgio Giacinto Fabio Roli
references.title: Evasion attacks against machine learning at test time”. In: Joint European conference on machine learning and knowledge discovery in databases (ECML-KDD)
references.authors: Adith Boloor Xin He Christopher Gill Yevgeniy Vorobeychik Xuan Zhang
references.title: Simple physical adversarial examples against end-to-end autonomous driving models
references.venue: IEEE International Conference on Embedded Software and Systems (ICESS)
references.authors: Tom B. Brown Dandelion Mané Aurko Roy Martín Abadi Justin Gilmer
references.title: Adversarial Patch. 2018
references.authors: Pin-Yu Chen Huan Zhang Yash Sharma Jinfeng Yi Cho-Jui Hsieh
references.title: Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models
references.venue: Workshop on Artificial Intelligence and Security
references.authors: Kevin Eykholt Ivan Evtimov Earlence Fernandes Bo Li Amir Rahmati Florian Tramer Atul Prakash Tadayoshi Kohno Dawn Song
references.title: Physical Adversarial Examples for Object Detectors
references.authors: Gamaleldin Elsayed Ian Goodfellow Jascha Sohl-Dickstein
references.title: Adversarial Reprogramming of Neural Networks
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Logan Engstrom Andrew Ilyas Shibani Santurkar Dimitris Tsipras Jacob Steinhardt Aleksander Madry
references.title: Identifying Statistical Bias in Dataset Replication
references.venue: In: International Conference on Machine Learning (ICML)
references.authors: Logan Engstrom Brandon Tran Dimitris Tsipras Ludwig Schmidt Aleksander Madry
references.title: Exploring the Landscape of Spatial Robustness
references.venue: In: International Conference on Machine Learning (ICML)
references.authors: Alhussein Fawzi Pascal Frossard
references.title: Manitest: Are classifiers really invariant?
references.venue: British Machine Vision Conference (BMVC)
references.authors: Mark Fiala
references.title: ARTag, a fiducial marker system using digital techniques
references.venue: IEEE Computer Society Conference on Computer Vision and Pattern Recognition
references.authors: Davide Falanga Alessio Zanchettin Alessandro Simovic Jeffrey Delmerico Davide Scaramuzza
references.title: Vision-based autonomous quadrotor landing on a moving platform
references.venue: IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR)
references.authors: Sergio Garrido-Jurado Rafael Muñoz-Salinas Francisco José Madrid-Cuevas Manuel Jesús
references.title: Marin-Jiménez. “Automatic generation and detection of highly reliable fiducial markers under occlusion
references.venue: Pattern Recognition
references.authors: Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix A. Wichmann Wieland Brendel
references.title: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang Evan Dorundo Rahul Desai Tyler Zhu Samyak Parajuli Mike Guo Dawn Song Jacob Steinhardt Justin Gilmer
references.title: The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
references.authors: Dan Hendrycks Thomas G. Dietterich
references.title: Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Andrew Ilyas Logan Engstrom Anish Athalye Jessy Lin
references.title: Black-box Adversarial Attacks with Limited Queries and Information
references.venue: In: International Conference on Machine Learning (ICML)
references.authors: Andrew Ilyas Logan Engstrom Aleksander Madry
references.title: Prior Convictions: Blackbox Adversarial Attacks with Bandits and Priors
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Andrew Ilyas Shibani Santurkar Dimitris Tsipras Logan Engstrom Brandon Tran Aleksander Madry
references.title: Adversarial Examples Are Not Bugs, They Are Features
references.venue: Neural Information Processing Systems (NeurIPS)
references.authors: Jorn-Henrik Jacobsen Jens Behrmann Richard Zemel Matthias Bethge
references.title: Excessive Invariance Causes Adversarial Vulnerability
references.venue: In: International Contemporary on Learning Representations
references.authors: Saumya Jetley Nicholas Lord Philip Torr
references.title: With friends like these, who needs adversaries?
references.venue: Advances in Neural Information Processing Systems (NeurIPS)
references.authors: Alexey Kurakin Ian J. Goodfellow Samy Bengio
references.title: Adversarial Machine Learning at Scale
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Can Kanbak Seyed-Mohsen Moosavi-Dezfooli Pascal Frossard
references.title: Geometric robustness of deep networks: analysis and improvement
references.venue: In: Conference on Computer Vision and Pattern Recognition
references.authors: Alex Krizhevsky
references.title: Learning Multiple Layers of Features from Tiny Images
references.venue: Technical report
references.authors: Daniel Kang Yi Sun Dan Hendrycks Tom Brown Jacob Steinhardt
references.title: Testing Robustness Against Unforeseen Adversaries
references.authors: Mark Lee Zico Kolter
references.title: On physical adversarial patches for object detection
references.venue: arXiv preprint arXiv:1906.11897
references.authors: Sven Lange Niko Sunderhauf Peter Protzel
references.title: A vision based onboard approach for landing and position control of an autonomous multirotor UAV in GPS-denied environments
references.venue: International Conference on Advanced Robotics
references.authors: Xuancen Liu Shifeng Zhang Jiayi Tian Longbin Liu
references.title: An Onboard VisionBased System for Autonomous Landing of a Low-Cost Quadrotor on a Novel Landing Pad
references.authors: Marco Melis Ambra Demontis Battista Biggio Gavin Brown Giorgio Fumera Fabio Roli
references.title: Is deep learning safe for robot vision? adversarial examples against the icub humanoid
references.venue: Proceedings of the IEEE International Conference on Computer Vision Workshops
references.authors: Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu
references.title: Towards deep learning models resistant to adversarial attacks
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Phong Ha Nguyen Muhammad Arsalan Ja Hyung Koo Rizwan Ali Naqvi Noi Quang Truong Kang Ryoung Park
references.title: LightDenseYOLO: A fast and accurate marker tracker for autonomous UAV landing by visible light camera sensor on drone
references.venue: Sensors
references.authors: Paarth Neekhara Shehzeen Hussain Shlomo Dubnov Farinaz Koushanfar
references.title: Adversarial Reprogramming of Text Classification Neural Networks
references.venue: Empirical Methods in Natural Language Processing (EMNLP)
references.authors: Merlin Nimier-David Delio Vicini Tizian Zeltner Wenzel Jakob
references.title: Mitsuba 2: A Retargetable Forward and Inverse Renderer
references.venue: In: Transactions on Graphics (Proceedings of SIGGRAPH Asia)
references.authors: Edwin Olson
references.title: AprilTag: A robust and flexible visual fiducial system
references.venue: IEEE International Conference on Robotics and Automation
references.authors: Nicolas Papernot Patrick McDaniel Ian Goodfellow Somesh Jha Z Berkay Celik Ananthram Swami
references.title: Practical black-box attacks against machine learning
references.venue: Asia Conference on Computer and Communications
references.authors: Nicolas Papernot Patrick McDaniel Ian Goodfellow
references.title: Transferability in Machine Learning: from Phenomena to Black-box Attacks using Adversarial Samples
references.venue: ArXiv preprint arXiv:1605.07277
references.authors: Olga Russakovsky Jia Deng Hao Su Jonathan Krause Sanjeev Satheesh Sean Ma Zhiheng Huang Andrej Karpathy Aditya Khosla Michael Bernstein Alexander C. Berg Li Fei-Fei
references.title: ImageNet Large Scale Visual Recognition Challenge
references.venue: In: International Journal of Computer Vision (IJCV)
references.authors: Francisco J Romero-Ramire Rafael Muñoz-Salinas Rafael Medina-Carnicer
references.title: Fractal Markers: a new approach for long-range marker pose estimation under occlusion
references.venue: IEEE Access
references.authors: Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar
references.title: Do ImageNet Classifiers Generalize to ImageNet?
references.venue: In: International Conference on Machine Learning (ICML)
references.authors: Aditi Raghunathan Sang Michael Xie Fanny Yang John C Duchi Percy Liang
references.title: Adversarial Training Can Hurt Generalization
references.authors: Mahmood Sharif Sruti Bhagavatula Lujo Bauer Michael K. Reiter
references.title: Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition
references.venue: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications
references.authors: Chawin Sitawarin Arjun Nitin Bhagoji Arsalan Mosenia Mung Chiang Prateek Mittal
references.title: Darts: Deceiving autonomous cars with toxic signs
references.authors: Hadi Salman Andrew Ilyas Logan Engstrom Ashish Kapoor Aleksander Madry
references.title: Do Adversarially Robust ImageNet Models Transfer Better?
references.venue: Advances in Neural Information Processing Systems (NeurIPS)
references.authors: Srikanth Saripalli James F Montgomery Gaurav S Sukhatme
references.title: Visually guided landing of an unmanned aerial vehicle
references.venue: IEEE transactions on robotics and automation
references.authors: Dong Su Huan Zhang Hongge Chen Jinfeng Yi Pin-Yu Chen Yupeng Gao
references.title: Is Robustness the Cost of Accuracy? A Comprehensive Study on the Robustness of 18 Deep Image Classification Models
references.venue: European Conference on Computer Vision (ECCV)
references.authors: Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow Rob Fergus
references.title: Intriguing properties of neural networks
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Rohan Taori Achal Dave Vaishaal Shankar Nicholas Carlini Benjamin Recht Ludwig Schmidt
references.title: Measuring Robustness to Natural Distribution Shifts in Image Classification
references.venue: Neural Information Processing Systems (NeurIPS)
references.authors: Antonio Torralba Alexei A Efros
references.title: Unbiased look at dataset bias
references.authors: Noi Quang Truong Phong Ha Nguyen Se Hyun Nam Kang Ryoung Park
references.title: Deep learning-based super-resolution reconstruction and marker detection for drone landing
references.venue: IEEE Access
references.authors: Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner Aleksander Madry
references.title: Robustness May Be at Odds with Accuracy
references.venue: In: International Conference on Learning Representations (ICLR)
references.authors: Simen Thys Wiebe Van Ranst Toon Goedemé
references.title: Fooling automated surveillance cameras: adversarial patches to attack person detection
references.venue: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
references.authors: John Wang Edwin Olson
references.title: AprilTag 2: Efficient and robust fiducial detection
references.venue: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
references.authors: D. Wang D. Tseng P. Li Y. Jiang M. Guo M. Danielczuk J. Mahler J. Ichnowski K. Goldberg
references.title: Adversarial Grasp Objects
references.venue: IEEE 15th International Conference on Automation Science and Engineering (CASE)
references.authors: Xiaojun Xu Xinyun Chen Chang Liu Anna Rohrbach Trevor Darrell Dawn Song
references.title: Fooling vision and language models despite localization and attention mechanism
references.venue: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
references.authors: Cihang Xie Jianyu Wang Zhishuai Zhang Yuyin Zhou Lingxi Xie Alan Yuille
references.title: Adversarial examples for semantic segmentation and object detection
references.venue: Proceedings of the IEEE International Conference on Computer Vision
references.authors: Shaowu Yang Sebastian A Scherer Konstantin Schauwecker Andreas Zell
references.title: Autonomous landing of MAVs on an arbitrarily textured landing site using onboard monocular vision
references.venue: Journal of Intelligent & Robotic Systems
sections.heading: 1 Introduction
sections.text: Performing reliably on unseen or shifting data distributions is a difficult challenge for modern computer vision systems. For example, slight rotations and translations of images suffice to reduce the accuracy of state-of-the-art classifiers [ETT+19; ALG+19; KMF18]. Similarly, models that attain near human-level performance on benchmarks exhibit significantly degraded performance when faced with even mild image corruptions and transformations [HD19; KSH+19]. In fact, when an adversary is allowed to modify inputs directly, standard vision models can be manipulated into predicting arbitrary outputs (cf. adversarial examples [BCM+13; SZS+14]). While robustness interventions and additional training data can improve out-of-distribution behavior, they do not fully close the gap between model performance on standard heldout data and on corrupted/otherwise unfamiliar data [TDS+20; HBM+20]. The situation is worse still when test-time distribution is underor mis-specified, which occurs commonly in practice.
  How can we change this state of affairs? We propose a new approach to image recognition in the face of unforeseen corruptions or distribution shifts. This approach is rooted in a reconsideration of the problem setup itself. Specifically, we observe that in many situations, a system designer actually controls, to some extent, the inputs that are fed into that model. For example, a drone operator seeking to train a landing pad detector can modify the surface of the landing pad; and, a roboticist training a perception model to recognize a small set of custom objects can slightly alter the texture or design of these objects.
  We find that such control over inputs can be leveraged to drastically improve our ability to tackle computer vision tasks. In particular, it allows us to turn the input-sensitivity of modern vision systems from a weakness into a strength. Instead of optimizing inputs to mislead models (e.g., as in adversarial examples), we can alter inputs to reinforce correct behavior, yielding what we refer to as “unadversarial examples.” Indeed, we show that even a simple gradient-based algorithm can
  ∗Equal contribution. †Work partially completed while at Microsoft Research. 1Our code is available at https://github.com/microsoft/unadversarial.
  35th Conference on Neural Information Processing Systems (NeurIPS 2021).
  successfully construct unadversarial examples in a variety of vision settings and demonstrate that, by optimizing objects for vision systems (rather than vice-versa), we can significantly improve both in-distribution performance and robustness to unforeseen data shifts and corruptions.
  We demonstrate the efficacy of our methods on both standard benchmarks (CIFAR, ImageNet) and robustness-based benchmarks (ImageNet-C, CIFAR-C) while also comparing them to a broad set of baselines (e.g., QR codes or heuristically designed patches). To further highlight the practicality of our framework, we (a) extend our methods to designing the texture of three-dimensional objects (rather than patches); (b) deploy unadversarial examples in a simulated drone setting; and (c) ensure that the performance improvement yielded by the objects we design actually transfer to the physical world.
sections.heading: 2 Motivation and approach
sections.text: While vision models tend to perform well on held-out data drawn from the same distribution as the training data, out-of-distribution inputs can severely degrade this performance. For example, models behave unreliably under distribution shifts induced by new data collection procedures [RRS+19; EIS+20; TE11], synthetic corruptions [HD19; KSH+19], spatial transformations [ETT+19; FF15], as well as under other types of shift.
  Given a fixed type of distribution shift, a standard approach to increasing model robustness is to explicitly train on or regularize with data from the corresponding anticipated test distribution [KSH+19]. For example, Engstrom et al. [ETT+19] find that vision models trained on worst-case rotations and translations end up being fairly robust to rotation and translation-based distribution shifts. However, this approach is not without shortcomings—for example, Kang et al. [KSH+19] find that training CIFAR classification models that are robust to JPEG-compression in this manner requires a significant sacrifice in natural accuracy. Recent works make similar observations in the context of other distribution shift mechanisms like `p adversaries [TSE+19; SZC+18; RXY+19] or texture swapping [GRM+19].
  These observations give rise to a more general question: given that performing reliably in the face of constrained, well-specified distribution shifts is already a difficult challenge, how can we attain robustness to broad, unforeseen distribution shifts?
sections.heading: 2.1 Leveraging more controlled vision settings
sections.text: Consider the vision tasks of detecting a landing pad from a drone, or classifying manufacturing components from a factory robot. In both these tasks, reliable in-distribution performance is a necessity; still, a number of possible distribution shifts may occur at deployment time. For example, the drone might approach the landing pad at an atypical angle, or have a view obstructed by snow,
  smoke, or rain. Similarly, the factory robot may encounter objects in unfamiliar poses, or could be equipped with only a low-quality/noisy camera.
  At first glance, dealing with these issues seems to require tackling the difficult problem of general distribution shift robustness discussed earlier in this section. However, there is in fact a critical distinction between the scenarios considered above and vision tasks in their full generality. In particular, in these scenarios and many others, the system designer has control over the physical objects that the model operates on. For instance, the designer of the drone’s landing algorithm could paint the landing pad bright yellow. A machine learning model trained to detect this custom landing pad might then be more effective than a model trained to detect a standard grey pad, especially in lowvisibility conditions. Still, the particular choice to paint the landing pad yellow is rather ad hoc, and likely rooted in the way humans recognize objects. Meanwhile, an abundance of prior work (e.g., [JBZ+19; GRM+19; JLT18; IST+19]) demonstrates that humans and machine learning models tend to use different sets of features to make their decisions. This suggests that rather than relying on human priors, we should instead be asking: how can we build objects that are easily detectable by machine learning models?
sections.heading: 2.2 Unadversarial examples
sections.text: The task of making inputs less recognizable by computer vision systems has been a focus of research in adversarial examples. Adversarial examples are small, carefully constructed perturbations to natural images that can induce arbitrary (mis)behavior from machine learning models [BCM+13; SZS+14]. These perturbations are typically constructed as the result of an optimization problem that maximizes the loss of a machine learning model with respect to the input, i.e., by solving the optimization problem
  δadv = arg max δ∈∆ L(fθ(x+ δ), y), (1)
  where fθ is a parameterized model (e.g., a neural network with weights θ); x is a natural input; y is the corresponding correct label; L is the loss function used to train θ (e.g., cross-entropy loss) and ∆ is a class of permissible perturbations (e.g., norm-bounded perturbations: ∆ = {δ : ‖δ‖p ≤ } for some small > 0). Adversarial perturbations are typically crafted via projected gradient descent (PGD) [Nes03] in input space, a standard iterative first-order optimization method—prior work in adversarial examples has shown that even a few iterations of PGD suffice to completely change the prediction of many state-of-the-art machine learning systems [MMS+18].
  From adversarial examples to unadversarial objects. The goal of this work is to modify the design of objects so that they are more easily recognizable by computer vision systems. If we could specify every pixel of every image that a model encounters at test time, we could draw on the effectiveness of adversarial examples, and construct image perturbations (using PGD) that minimize the loss of the system, e.g.,
  δunadv = arg min δ∈∆ L(θ;x+ δ, y). (2)
  In our setting of interest, however, having such fine-grained access to the test inputs is unrealistic (presumably, if we had precise control over every pixel in the input, we could just directly encode the ground-truth label directly in it). Instead, we have limited control over some physical objects; these objects are in turn captured within images, affected by many signals that are out of our control, such as camera artifacts, weather effects, or background scenery.
  It turns out that we can still draw on techniques from adversarial examples research in this limitedcontrol setting. Specifically, a recent line of work [KGB17; SBB+16; EEF+18a; AEI+18] concerns itself with constructing robust adversarial examples [AEI+18], i.e., physically realizable objects that act as adversarial examples when introduced into a scene in any one of a variety of ways. For example, Sharif et al. [SBB+16] design glasses frames that cause facial recognition models to misclassify faces, Athalye et al. [AEI+18] design custom-textured 3D models that are misclassified by state-of-the-art ImageNet classifiers from many angles and viewpoints, and [BMR+18] design adversarial patches: stickers that can be placed anywhere on objects causing them to be misclassified. In this paper, we leverage the techniques developed in the above line of work to construct robust un-adversarial objects—physically realizable objects optimized to minimize (rather than maximize) the loss of a target classifier. In the next section, we will more concretely discuss our methods for generating unadversarial objects, then outline our evaluation setup.
sections.heading: 2.3 Constructing unadversarial objects
sections.text: In the previous section, we identified a class of scenarios where a system designer can, to some extent, control the objects that a machine learning system model operates on. In these settings, we motivated so-called unadversarial examples as a potential way to boost models’ overall performance and robustness to distribution shifts. In this section, we present and illustrate two concrete algorithms for constructing unadversarial examples: unadversarial patches and unadversarial textures. In the former, we design a sticker or “patch” [BMR+18] that can be placed on the object; in the latter, we design the 3D texture of the object (in a similar manner to the texture-based adversarial examples of Athalye et al. [AEI+18]). Example results from both techniques are shown in Figure 2. For simplicity, we will assume that the task being performed is image classification, but the techniques are directly applicable to other tasks as well. In all cases, we require access to a pre-trained model for the dataset of interest.
  Unadversarial patches (cf. Algorithm 1 in Appendix A). To train unadversarial patches (cf. Figure 2a), in addition to the pre-trained model, we require sample access to image-label pairs from the dataset of interest. At each iteration, we sample an image-label pair (x, y) from a training set, and place the patch corresponding to class y onto the image with random orientation and position2. Since placing the patch is an affine transformation, after each iteration we can compute the gradient of the model’s loss with respect to the pixels in the patch, and take a negative gradient step on the patch parameters. The algorithm terminates when the model’s loss on sticker-boosted images plateaus, or after a fixed number of iterations.
  Unadversarial textures (cf. Algorithm 2 in Appendix A). To train unadversarial textures (cf. Figure 2b), we do not require sample access to the dataset, but instead a set of 3D meshes for each class of objects that we would like to augment, as well as a set of background images that we can use to simulate sampling a scene (these can be images from the dataset of interest, solid-color backgrounds, random patterns, etc.).
  For each 3D mesh, our goal is to optimize a 2D texture which improves classifier performance when mapped onto the mesh. At each iteration, we sample a mesh and a random background; we then use a 3D renderer (Mitsuba [NVZ+19]) to map the corresponding texture onto the mesh. We overlay the rendering onto a random background image, and then feed the resulting composed image into the pre-trained classifier, with the label being that of the sampled 3D mesh. Since rendering is typically non-differentiable, we use a linear approximation of the rendering process (cf. Athalye et al. [AEI+18]) in order to compute (this time approximate) gradients of the model’s loss with respect to the utilized texture. From there, we apply the same SGD algorithm as we did for the patch case.
sections.heading: 3 Experimental evaluation
sections.text: In order to determine the effectiveness of our proposed framework, we evaluate against a suite of computer vision tasks. Below, we first provide some detail on the precise access model and baselines that will be considered. We then briefly outline the experimental setup of each task, and show that
  2We allow the patch to be placed anywhere as a matter of convenience: ideally we would only be applying the patch onto the main object itself, but this would require bounding box data that we do not have for most classification datasets.
  unadversarial objects consistently improve the performance and robustness of the vision systems tested. For a more detailed account of each experimental setup, see Appendix C.
sections.heading: 3.1 Access model and baselines
sections.text: In many of the settings discussed thus far, a system designer can alter the objects being recognized but is not allowed to alter the the classifier itself. That is, we are optimizing unadversarial objects for a fixed (pre-trained) model. For instance, a road engineer may wish to design road signs that are easier to recognize for autonomous vehicles, without being able to train or alter the machine learning models that operate the vehicles. Similarly, a roboticist might want to design a landing pad that works better for a commercial (pre-trained) drone. We will refer to this setting as the fixedmodel setting. On the other hand, sometimes the same entity is able to train both the model and the transformations (the discussed factory robot example may fall into this category, for example). In this “free-model” setting, one may be able to boost performance by co-designing the machine learning model and the objects of interest.
  In this work, we will focus on designing unadversarial examples in the fixed-model setting, for the sake of both simplicity and applicability. In particular, any valid algorithm under a fixed-model assumption is also a valid algorithm under the co-design assumption (but the converse is not true). This leaves the task of leveraging even more control in joint optimization settings as a potential avenue for future work.
  Baselines. Since we consider the fixed-model setting throughout our paper, the only truly comparable baselines are those which do not alter the model being trained. Nonetheless, in order to fully contextualize our results, we will also consider a few baselines that fall outside of our intended access model (e.g., QR codes). A notable disadvantage of these baselines (that we do not explicitly
  demonstrate below) is that if the “unadversarial signal” (e.g., the QR code) is occluded or removed, the entire system fails; this is in contrast to the fixed-model setting, where the pre-trained model is able to recognize objects without any unadversarial signals and is instead “boosted” by their presence.
sections.heading: 3.2 Clean data and synthetic corruptions
sections.text: We first test whether unadversarial examples improve the performance of image classifiers on benchmark datasets. Using the algorithm described in Section 2.3, we construct unadversarial patches of varying size for pre-trained ResNet-50 classifiers on the CIFAR [Kri09] and ImageNet [RDS+15] datasets. For evaluation, we add these patches at random positions, scales, and orientations to validation set images (see Appendix C for the exact protocol). As shown in Figure 4a, the pre-trained ImageNet classifier is consistently more accurate on the augmented ImageNet images. For example, an unadversarial patch 20 times smaller than ImageNet images boosts accuracy by 26.3% (analogous results for CIFAR are given in Appendix D).
  Robustness to synthetic corruptions. Next, we use the CIFAR-C and ImageNet-C datasets [HD19] (consisting of the CIFAR and ImageNet validation sets corrupted in 15 systematic ways) to see whether the addition of unadversarial patches to images confers any corruption robustness.
  We use the same patches and evaluation protocol that we used when looking at clean data (to ensure a fair evaluation, we apply corruptions to boosted images only after the unadversarial patches have been applied). As a consequence, at test time neither model nor patch has been exposed to any image corruptions beyond standard data augmentation. As a result, this experiment tests the ability for unadversarially boosted images to withstand completely unforeseen corruptions; we also avoid any potential biases from training on (and thus “overfitting” to [KSH+19]) a specific type of corruption. The results (cf. Figure 4b for ImageNet and Appendix D for CIFAR) indicate that unadversarial patches do improve performance across corruption types; for example, applying an unadversarial patch 5% the size of a standard ImageNet image boosts accuracy by an average of 31.7% points across corruptions 3.
  The model does not ignore the image in the presence of unadversarial patches.
  Recall from our discussion of the fixed-model setting in Section 3.1 that an advantage of designing unadversarial objects without changing the model is that the model still works in the absence of the unadversarial signal. We now briefly explore the case where the model is exposed to an unadversarial signal for the wrong class. Ideally, we would want the patch to only assist/boost the signal from the original image—in particular, we do not want the patch to make the model totally ignore the contents of the image itself. Thus, in cases where the signal from the image and the patch conflict, we would like the classifier to predict according to the features present in the image more frequently than the class encoded in the unadversarial patch.
  3Since the original corruption benchmarks proposed by [HD19] are only available as pre-computed JPEGs (for which we cannot apply a patch pre-corruption) or CPU-based Python image operations (which were prohibitively slow), we re-implemented all 15 corruptions as batched GPU operations; we verified that model accuracies on our corruptions mirrored the original CPU counterparts (i.e., within 1% accuracy). For more details about our reimplementation, see our code release.
  Indeed, this turns out to be the case. Figure 5 shows that on CIFAR-10, when the signal from an unadversarial patch and the image itself conflict, the model predicts according to the patch only 31.2% of the time on average, and according to the image 60.3% of the time (the accuracy of the model when the patch and image agree is 98.93%.)
  At first glance, this result may seem to be at odds with the near-perfect effectiveness of adversarial patches [BMR+18]. However, the phenomenon we observe here can be tied to the subtle difference between the way we train our unadversarial patches and the way one trains targeted adversarial patches. In the former, we overlay each patch exclusively onto images from its respective class— thus, unadversarial patches are never optimized to be effective when overlaid on a different class. In the latter, however, adversarial patches are optimized to maximize confidence in a particular class on all possible backgrounds, making the patch dominant even when overlaid on an image from a different class.
  Baselines. We also compare our results to a variety of natural baselines; the most relevant of these is the “best loss image patch,” where we use the minimum-loss training image in place of a patch. We compare with this baseline to ensure that our method is doing something beyond this naive way to add signal to an image. The results are shown in Appendix D, along with comparisons to less sophisticated baselines, such as QR Codes and predefined random Gaussian noise patches.
sections.heading: 3.3 Classification in 3D simulation
sections.text: We now test unadversarial examples in a more practical setting: recognizing 3D objects in a high-fidelity simulator. We collect meshes corresponding to four ImageNet classes: “warplane,” “minibus,” “container ship,” and “trailer truck,” from sketchfab.com. We generate a texture for each object using the unadversarial texture algorithm of Section 2.3, using the ImageNet validation set as the set of backgrounds for the algorithm, and a pre-trained ResNet-50 as the classifier.
  To evaluate the resulting textures, we import each mesh into Microsoft AirSim, a high-fidelity threedimensional simulator; we then test pre-trained ImageNet models’ ability to recognize each object with and without the unadversarial texture applied in a variety of surroundings. We also test each texture’s robustness to more realistic weather corruptions (snow and fog) built directly into the simulator (rather than applied as a post-processing step). We provide further detail on AirSim and our usage of it in Appendix B. Examples of the images used to evaluate the unadversarial textures, as well as our main results for one of the meshes are shown in Figure 6. We find that in both standard and adverse weather conditions, the model consistently performs better on the unadversarial texture than on the original. We present similar results for the other three meshes in Figure 7.
sections.heading: 3.4 Localization for (simulated) drone landing
sections.text: We then assess whether unadversarial examples can help outside of the classification setting. Again using AirSim, we set up a drone landing task with a perception module that receives as input an axis-aligned aerial image of a landing pad, and is tasked with outputing an estimate of the camera’s (x, y)-position relative to the pad. While this task is quite basic, we are particularly interested in studying performance in the presence of heavy (simulated) weather-based corruptions. The drone is equipped with a pretrained regression model that localizes the landing pad (described in detail in Appendix B). We optimize an unadversarial texture for the surface of the landing pad to best help the drone’s regression model in localization. Figure 8 shows an example of the landing pad localization task, along with the performance of the unadversarial landing pad compared to the standard pad. The drone landing on the unadversarial pad consistently lands both more reliably.
sections.heading: 3.5 Physical-world unadversarial examples
sections.text: Finally, we move out of simulation and test whether the unadversarial patches that we generate can survive naturally-arising distribution shift from effects such as real lighting, camera artifacts, and printing imperfections. We use four household objects (a toy racecar, miniature plane, coffeepot, and eggnog container), and print out (on a standard InkJet printer) the adversarial patch corresponding to the label of each object. We take pictures of the toy with and without the patch taped on using an ordinary cellphone camera, and count the number of poses for which the toy is correctly classified by a pre-trained ImageNet classifier. Our results are in Table 9a, and examples of patches are in Figure 9b. Classifying both patched and unpatched images over a diverse set of poses, we find that the adversarial patches consistently improve performance even at uncommon object orientations.
sections.heading: 4 Related work
sections.text: Here, we first highlight (and differentiate from) previous works using reference markers to improve recognition and localization. We will then discuss related work in adversarial robustness.
  Improving computer vision with fiducial markers. In past research, vision-based precision landing was initially attempted with classical computer vision based tracking of reference designs
  [SMS03; LSP09; YSS+14; FZS+17], or on arrangements of fiducial markers [LZT+19] (a fiducial marker is a fixed pattern or object that is placed in a scene as a reference point for location or measure). Several types of visual fiducial markers were also proposed for robust tracking and pose estimation applications through classical vision [Fia05; GMM+14; Ols11; RMM19]. While fiducial markers such as AprilTags [Ols11] were commonly used in robotics, they suffer from limitations such as orientation uncertainty, accuracy falloff depending on viewing angles, and short detection ranges [AAB+19; WO16], which motivated research into using convolutional neural networks for landing pad detection and pose estimation [NAK+18; TNN+19]. Our work attempts to unify these perspectives by leveraging the expressivity of neural network-based systems to design robust unadversarial examples. Another key difference between fiducial markers and unadversarial objects is that the former require vision systems to be aware of their presence; in contrast, the latter are designed on top of pre-trained systems. This means that we do not require any further non-standard model training, and that we do not depend as heavily on the unadversarial example being visible.
  Adversarial robustness. Our design of unadversarial examples is motivated by the success of adversarial examples, i.e., minute perturbations to the inputs of machine learning models that can induce nearly arbitrary behavior. Adversarial examples were shown to be effective in seriously hampering machine vision related tasks such as classification [SZS+14; EEF+18a; AEI+18], object detection/segmentation [EEF+18b; AMT18; LK19; XWZ+17] and visual question-answering (VQA) [XCL+18]. Furthermore, prior work has shown that adversarial examples can be constructed even without direct access to the vision system being manipulated [CZS+17; PMG16; PMG+17; IEA+18; IEM18] Synthesized physical adversarial examples were shown to be effective in fooling person detection [TVG19], sign detection for autonomous driving [SBM+18; BHG+19]. Robotic platforms such as manipulators were also shown to be sensitive to vision based adversarial examples [MDB+17], and to specifically designed adversarial objects [WTL+19]. Additionally, recent work [EGS19; NHD+19] shows that one can “reprogram” neural networks using adversarial examples, e.g., one can construct a patch that causes a CIFAR-10 classifier to operate as an MNIST classifier.
sections.heading: 5 Discussion and conclusions
sections.text: In this work, we demonstrated that it is possible to design object alternations that boost the corresponding classifiers’ performance, even under strong and unforeseen distribution shift. Indeed, such resulting unadversarial objects are robust to a broad range of data shifts and corruptions, even when these were never seen in training. We view our results as a promising route towards increasing out-of-distribution robustness of computer vision models.
  Limitations and future work. One limitation of our method is that it requires differentiability with respect to properties of the input object of interest, or a close proxy (e.g. a differentiable simulator) that can mimick the object and the environment in which the object operates.
  Furthermore, as discussed at length in Section 3.1, in this work we only consider the case where the system designer cannot alter the model being trained. While this carries some inherent advantages (e.g., increased applicability), future work should investigate whether attaining even more accuracy and robustness is possible by jointly optimizing models and unadversarial objects.
  Finally, we have shown unadversarial examples to be effective only in classification and regression settings. However, the fact that unadversarial examples and adversarial examples share the same underlying generation technique is evidence that unadversarial examples could apply to any system that is vulnerable to adversarial examples. Thus, an avenue of future work could be to apply these examples to detection, segmentation, style transfer, and other domains.
  Broader impact. On one hand, as with any improvement to the robustness and reliability of computer vision systems, unadversarial examples also carry the potential for misuse (e.g., as part of overreaching surveillance technology). On the other hand, unadversarial examples give end users of machine learning systems—who typically have little to no power over the behaviour of the ML systems they interact with—the power to improve the performance and robustness of these systems in their respective use cases. For example, in the future someone living in a remote area where roads or traffic signs may look “atypical” can slightly modify their signs to be better recognized by autonomous vehicles.
sections.heading: Acknowledgements
sections.text: We are grateful to Ian Engstrom for helping take the photographs for the physical-world experiments.
  Work supported in part by the NSF grants CCF-1553428 and CNS-1815221, Open Philanthropy, and the Microsoft Corporation. This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001120C0015.
  Research was sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
title: Unadversarial Examples: Designing Objects for Robust Vision