summaries: The paper extends current adversarial learning approaches beyond imperceptible L_p norm perturbations. The proposed approach can handle many models of natural variation, such as a change in brightness. The main idea behind the approach is to use unsupervised approaches such as GANs to model the natural variation. Given this model of natural variation, the paper replaces the adversarial learning objective of finding the worst example in an L_p norm ball around a data point to finding the worst example based on the model of natural variation. This is expensive, so the paper also proposes more computationally efficient approaches based on data augmentation. The experimental results demonstrate that the proposed approach performs well on a variety of tasks.

summaries: This paper proposes a model-based framework for improving the robustness of image classifiers to average-case corruptions of varying severity. The proposed framework can be thought of as adversarial training where the perturbation is replaced by a function that transforms the image according to a specific corruption. A nuisance parameter controls the instantiation and severity of the corruption that is applied to the input. The paper compares baselines to different versions of this general model-based framework with experiments on several datasets.

summaries: This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset.

abstractText: While deep learning (DL) has resulted in major breakthroughs in many applications, the frameworks commonly used in DL remain fragile to seemingly innocuous changes in the data. In response, adversarial training has emerged as a principled approach for improving the robustness of DL against norm-bounded perturbations. Despite this progress, DL is also known to be fragile to unbounded shifts in the data distribution due to many forms of natural variation, including changes in weather or lighting in images. However, there are remarkably few techniques that can address robustness to natural, out-of-distribution shifts in the data distribution in a general context. To address this gap, we propose a paradigm shift from perturbation-based adversarial robustness to model-based robust deep learning. Critical to our paradigm is to obtain models of natural variation, which vary data over a range of natural conditions. Then by exploiting these models, we develop three novel model-based robust training algorithms that improve the robustness of DL with respect to natural variation. Our extensive experiments show that across a variety of natural conditions in twelve distinct datasets, classifiers trained with our algorithms significantly outperform classifiers trained via ERM, adversarial training, and domain adaptation techniques. Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy. Further, we show that our methods provide robustness (1) against natural, out-ofdistribution data, (2) against multiple simultaneous distributional shifts, and (3) to domains entirely unseen during training.

authors:
Amjad Almahairi
Sai Rajeswar
Alessandro Sordoni
Philip Bachman
Aaron Courville

title: Augmented cyclegan: Learning many-to-many mappings from unpaired data

venue: arXiv preprint arXiv:1802.10151,

authors:
Vinicius F Arruda
Thiago M Paixão
Rodrigo F Berriel
Alberto F De Souza
Claudine Badue
Nicu Sebe
Thiago Oliveira-Santos

title: Cross-domain car detection using unsupervised image-toimage translation: From day to night

venue: In 2019 International Joint Conference on Neural Networks (IJCNN),

authors:
Anish Athalye
Nicholas Carlini
David Wagner

title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

venue: arXiv preprint arXiv:1802.00420,

authors:
Alesia Chernikova
Alina Oprea
Cristina Nita-Rotaru
BaekGyu Kim

title: Are self-driving cars secure? evasion attacks against deep neural networks for steering angle prediction

venue: IEEE Security and Privacy Workshops (SPW),

authors:
Yunjey Choi
Minje Choi
Munyoung Kim
Jung-Woo Ha
Sunghun Kim
Jaegul Choo

title: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation

venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,

authors:
Yunjey Choi
Youngjung Uh
Jaejun Yoo
Jung-Woo Ha

title: Stargan v2: Diverse image synthesis for multiple domains

venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,

authors:
Tarin Clanuwat
Mikel Bober-Irizar
Asanobu Kitamoto
Alex Lamb
Kazuaki Yamamoto
David Ha

title: Deep learning for classical japanese literature

venue: arXiv preprint arXiv:1812.01718,

authors:
Gregory Cohen
Saeed Afshar
Jonathan Tapson
Andre Van Schaik

title: Emnist: Extending mnist to handwritten letters

venue: In 2017 International Joint Conference on Neural Networks (IJCNN),

authors:
Jia Deng
Wei Dong
Richard Socher
Li-Jia Li
Kai Li
Li Fei-Fei

title: Imagenet: A large-scale hierarchical image database

venue: IEEE conference on computer vision and pattern recognition,

authors:
Josip Djolonga
Jessica Yung
Michael Tschannen
Rob Romijnders
Lucas Beyer
Alexander Kolesnikov
Joan Puigcerver
Matthias Minderer
Alexander D’Amour
Dan Moldovan

title: On robustness and transferability of convolutional neural networks

venue: arXiv preprint arXiv:2007.08558,

authors:
Edgar Dobriban
Hamed Hassani
David Hong
Alexander Robey

title: Provable tradeoffs in adversarially robust classification

venue: arXiv preprint arXiv:2006.05161,

authors:
Tommaso Dreossi
Alexandre Donzé
Sanjit A Seshia

title: Compositional falsification of cyberphysical systems with machine learning components

venue: Journal of Automated Reasoning,

authors:
Isaac Dunn
Tom Melham
Daniel Kroening

title: Generating realistic unrestricted adversarial inputs using dual-objective gan training

venue: arXiv preprint arXiv:1905.02463,

authors:
Logan Engstrom
Brandon Tran
Dimitris Tsipras
Ludwig Schmidt
Aleksander Madry

title: Exploring the landscape of spatial robustness

venue: arXiv preprint arXiv:1712.02779,

authors:
Kevin Eykholt
Ivan Evtimov
Earlence Fernandes
Bo Li
Amir Rahmati
Chaowei Xiao
Atul Prakash
Tadayoshi Kohno
Dawn Song

title: Robust physical-world attacks on deep learning visual classification

venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,

authors:
Mahyar Fazlyab
Manfred Morari
George J Pappas

title: Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming

venue: arXiv preprint arXiv:1903.01287,

authors:
Mahyar Fazlyab
Alexander Robey
Hamed Hassani
Manfred Morari
George Pappas

title: Efficient and accurate estimation of lipschitz constants for deep neural networks

venue: In Advances in Neural Information Processing Systems,

authors:
Yaroslav Ganin
Victor Lempitsky

title: Unsupervised domain adaptation by backpropagation

venue: In International conference on machine learning,

authors:
Yaroslav Ganin
Evgeniya Ustinova
Hana Ajakan
Pascal Germain
Hugo Larochelle
François Laviolette
Mario Marchand
Victor Lempitsky

title: Domain-adversarial training of neural networks

venue: The Journal of Machine Learning Research,

authors:
Ian Goodfellow
Jean Pouget-Abadie
Mehdi Mirza
Bing Xu
David Warde-Farley
Sherjil Ozair
Aaron Courville
Yoshua Bengio

title: Generative adversarial nets

venue: In Advances in neural information processing systems,

authors:
Ian J Goodfellow
Jonathon Shlens
Christian Szegedy

title: Explaining and harnessing adversarial examples

venue: arXiv preprint arXiv:1412.6572,

authors:
Sven Gowal
Chongli Qin
Po-Sen Huang
Taylan Cemgil
Krishnamurthy Dvijotham
Timothy Mann
Pushmeet Kohli

title: Achieving robustness in the wild via adversarial mixing with disentangled representations

authors:
Kaiming He
Xiangyu Zhang
Shaoqing Ren
Jian Sun

title: Deep residual learning for image recognition

authors:
Dan Hendrycks
Thomas Dietterich

title: Benchmarking neural network robustness to common corruptions and perturbations

venue: arXiv preprint arXiv:1903.12261,

authors:
Dan Hendrycks
Norman Mu
Ekin D Cubuk
Barret Zoph
Justin Gilmer
Balaji Lakshminarayanan

title: Augmix: A simple data processing method to improve robustness and uncertainty

venue: arXiv preprint arXiv:1912.02781,

authors:
Dan Hendrycks
Kevin Zhao
Steven Basart
Jacob Steinhardt
Dawn Song

title: Natural adversarial examples

venue: arXiv preprint arXiv:1907.07174,

authors:
Dan Hendrycks
Steven Basart
Norman Mu
Saurav Kadavath
Frank Wang
Evan Dorundo
Rahul Desai
Tyler Zhu
Samyak Parajuli
Mike Guo

title: The many faces of robustness: A critical analysis of out-of-distribution generalization

venue: arXiv preprint arXiv:2006.16241,

authors:
Hossein Hosseini
Radha Poovendran

title: Semantic adversarial examples

venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,

authors:
Xun Huang
Ming-Yu Liu
Serge Belongie
Jan Kautz

title: Multimodal unsupervised image-toimage translation

venue: In Proceedings of the European Conference on Computer Vision (ECCV),

title: A database for handwritten text recognition research

venue: IEEE Transactions on pattern analysis and machine intelligence,

authors:
Ajil Jalal
Andrew Ilyas
Constantinos Daskalakis
Alexandros G Dimakis

title: The robust manifold defense: Adversarial training using generative models

venue: arXiv preprint arXiv:1712.09196,

authors:
Sandesh Kamath
Amit Deshpande
KV Subrahmanyam

title: Invariance vs. robustness of neural networks

venue: arXiv preprint arXiv:2002.11318,

authors:
Nikolaos Karianakis
Jingming Dong
Stefano Soatto

title: An empirical evaluation of current convolutional architectures’ ability to manage nuisance location and scale variability

authors:
Diederik P Kingma
Jimmy Ba

title: Adam: A method for stochastic optimization

venue: arXiv preprint arXiv:1412.6980,

authors:
Jack Klys
Jake Snell
Richard Zemel

title: Learning latent subspaces in variational autoencoders

authors:
Alexey Kurakin
Ian Goodfellow
Samy Bengio

title: Adversarial examples in the physical world

venue: arXiv preprint arXiv:1607.02533,

authors:
Yann LeCun
Corinna Cortes
CJ Burges

title: Mnist handwritten digit database

venue: ATT Labs [Online]. Available: http://yann. lecun. com/exdb/mnist,

authors:
Hsin-Ying Lee
Hung-Yu Tseng
Jia-Bin Huang
Maneesh Singh
Ming-Hsuan Yang

title: Diverse image-to-image translation via disentangled representations

venue: In Proceedings of the European conference on computer vision (ECCV),

authors:
Hyeungill Lee
Sungyeob Han
Jungwoo Lee

title: Generative adversarial trainer: Defense to adversarial perturbations with gan

venue: arXiv preprint arXiv:1705.03387,

authors:
Dan Li
Dacheng Chen
Baihong Jin
Lei Shi
Jonathan Goh
See-Kiong Ng

title: Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks

venue: In International Conference on Artificial Neural Networks,

authors:
Ming-Yu Liu
Thomas Breuel
Jan Kautz

title: Unsupervised image-to-image translation networks

authors:
Aleksander Madry
Aleksandar Makelov
Ludwig Schmidt
Dimitris Tsipras
Adrian Vladu

title: Towards deep learning models resistant to adversarial attacks

venue: arXiv preprint arXiv:1706.06083,

authors:
Qi Mao
Hsin-Ying Lee
Hung-Yu Tseng
Siwei Ma
Ming-Hsuan Yang

title: Mode seeking generative adversarial networks for diverse image synthesis

authors:
Marco Melis
Ambra Demontis
Battista Biggio
Gavin Brown
Giorgio Fumera
Fabio Roli

title: Is deep learning safe for robot vision? adversarial examples against the icub humanoid

venue: In Proceedings of the IEEE International Conference on Computer Vision Workshops,

authors:
Yuval Netzer
Tao Wang
Adam Coates
Alessandro Bissacco
Bo Wu
Andrew Y Ng

title: Reading digits in natural images with unsupervised feature learning

venue: NIPS Workshop on Deep Learning and Unsupervised Feature Learning,

authors:
Konstantinos Papangelou
Konstantinos Sechidis
James Weatherall
Gavin Brown

title: Toward an understanding of adversarial examples in clinical trials

venue: In Joint European Conference on Machine Learning and Knowledge Discovery in Databases,

authors:
Kexin Pei
Yinzhi Cao
Junfeng Yang
Suman Jana

title: Deepxplore: Automated whitebox testing of deep learning systems

venue: In proceedings of the 26th Symposium on Operating Systems Principles,

authors:
Aditi Raghunathan
Jacob Steinhardt
Percy Liang

title: Certified defenses against adversarial examples

venue: arXiv preprint arXiv:1801.09344,

authors:
Pouya Samangouei
Maya Kabkab
Rama Chellappa

title: Defense-gan: Protecting classifiers against adversarial attacks using generative models

venue: arXiv preprint arXiv:1805.06605,

authors:
Yang Song
Rui Shu
Nate Kushman
Stefano Ermon

title: Constructing unrestricted adversarial examples with generative models

authors:
Johannes Stallkamp
Marc Schlipsing
Jan Salmen
Christian Igel

title: The German Traffic Sign Recognition Benchmark: A multi-class classification competition

venue: In IEEE International Joint Conference on Neural Networks,

authors:
Christian Szegedy
Wojciech Zaremba
Ilya Sutskever
Joan Bruna
Dumitru Erhan
Ian Goodfellow
Rob Fergus

title: Intriguing properties of neural networks

venue: arXiv preprint arXiv:1312.6199,

authors:
Rohan Taori
Achal Dave
Vaishaal Shankar
Nicholas Carlini
Benjamin Recht
Ludwig Schmidt

title: Measuring robustness to natural distribution shifts in image classification

venue: arXiv preprint arXiv:2007.00644,

authors:
Dogancan Temel
Min-Hung Chen
Ghassan AlRegib

title: Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics

venue: IEEE Transactions on Intelligent Transportation Systems,

authors:
Eric Tzeng
Judy Hoffman
Kate Saenko
Trevor Darrell

title: Adversarial discriminative domain adaptation

authors:
Simon Vandenhende
Bert De Brabandere
Davy Neven
Luc Van Gool

title: A three-player gan: generating hard samples to improve classification networks

venue: In 2019 16th International Conference on Machine Vision Applications (MVA),

authors:
Huaxia Wang
Chun-Nam Yu

title: A direct approach to robust deep learning using adversarial networks

venue: arXiv preprint arXiv:1905.09591,

authors:
Mei Wang
Weihong Deng

title: Deep visual domain adaptation: A survey

authors:
Ze Wang
Xiuyuan Cheng
Guillermo Sapiro
Qiang Qiu

title: Stochastic conditional generative networks with basis decomposition

venue: arXiv preprint arXiv:1909.11286,

authors:
Eric Wong
J Zico Kolter

title: Provable Defenses Against Adversarial Examples Via the Convex Outer Adversarial Polytope

venue: arXiv preprint arXiv:1711.00851,

title: Learning perturbation sets for robust machine learning

venue: arXiv preprint arXiv:2007.08450,

authors:
Chaowei Xiao
Bo Li
Jun-Yan Zhu
Warren He
Mingyan Liu
Dawn Song

title: Generating adversarial examples with adversarial networks

venue: arXiv preprint arXiv:1801.02610,

authors:
Chaowei Xiao
Jun-Yan Zhu
Bo Li
Warren He
Mingyan Liu
Dawn Song

title: Spatially transformed adversarial examples

venue: arXiv preprint arXiv:1801.02612,

authors:
Han Xiao
Kashif Rasul
Roland Vollgraf

title: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

venue: arXiv preprint arXiv:1708.07747,

authors:
Chhavi Yadav
Léon Bottou

title: Cold case: The lost mnist digits

authors:
Zili Yi
Hao Zhang
Ping Tan
Minglun Gong

title: Dualgan: Unsupervised dual learning for imageto-image translation

venue: In Proceedings of the IEEE international conference on computer vision,

title: Adadelta: an adaptive learning rate method

venue: arXiv preprint arXiv:1212.5701,

authors:
Jun-Yan Zhu
Taesung Park
Phillip Isola
Alexei A Efros

title: Unpaired image-to-image translation using cycle-consistent adversarial networks

authors:
Jun-Yan Zhu
Richard Zhang
Deepak Pathak
Trevor Darrell
Alexei A Efros
Oliver Wang
Eli Shechtman

title: Toward multimodal image-to-image translation

heading: 1 INTRODUCTION

text:
The last decade has seen remarkable progress in deep learning (DL), which has prompted widescale integration of DL frameworks into myriad application domains (LeCun et al., 2015). In many of these applications, and in particular in safety-critical domains, it is essential that the DL systems are robust and trustworthy (Dreossi et al., 2019). However, it is now well-known that DL is fragile to seemingly innocuous changes to the input data (Szegedy et al., 2013). Indeed, well-documented examples of fragility to carefully-designed noise can be found in a variety of contexts, including image classification (Madry et al., 2017), clinical trials (Papangelou et al., 2018), and robotics (Melis et al., 2017). Accordingly, a number of adversarial training algorithms (Goodfellow et al., 2014b; Wong & Kolter, 2017) as well as certifiable defenses (Raghunathan et al., 2018; Fazlyab et al., 2019a) have recently been proposed, which have provided a rigorous framework for improving the robustness of DL against norm-bounded perturbations (Fazlyab et al., 2019b; Dobriban et al., 2020).
Despite this encouraging progress, very recent papers have unanimously shown that DL is also fragile to unbounded shifts in the data-distribution due to a wide range of natural phenomena (Djolonga et al., 2020; Taori et al., 2020; Hendrycks et al., 2020; Hendrycks & Dietterich, 2019). For example, in image classification, such shifts include changes due to lighting, blurring, or weather conditions (Pei et al., 2017; Chernikova et al., 2019). However, there are remarkably few general, principled techniques that have been shown to provide robustness against these forms of out-of-distribution, natural variation (Hendrycks et al., 2019a). Furthermore, as these unseen distributional shifts are arguably more common in safety-critical domains, the task of designing algorithms that generalize to natural, out-of-distribution data is an important and novel challenge for the DL community.
In this paper, we propose a paradigm shift from perturbation-based adversarial robustness to modelbased robust deep learning. Our goal is to provide principled, general algorithms that can be used to train neural networks to be robust against natural, out-of-distribution shifts in data. Our experiments show that across a variety of challenging, naturally-occurring conditions, such as variation in lighting, haze, rain, and snow, and across various datasets, including SVHN, GTSRB, CURE-TSR, and ImageNet, classifiers trained with our model-based algorithms significantly outperform standard DL baselines, adversarially-trained classifiers, and, when applicable, domain adaptation methods.
Contributions. The contributions of this paper can be summarized as follows: • Paradigm shift. We propose a paradigm shift from perturbation-based robustness to model-based
robust deep learning, where models of natural variation express changes due to natural conditions. • Optimization-based formulation. We formulate a novel model-based robust training problem by
constructing a general robust optimization procedure to search for challenging natural variation. • Models of natural variation. For many challenging forms of natural variation, we use deep
generative models to learn models of natural variation that are consistent with realistic conditions. • Novel algorithms. We propose a family of novel robust training algorithms that exploit models
of natural variation to improve the robustness of DL against worst-case natural variation. • Out-of-distribution robustness. We show that our algorithms are the first to consistently provide
robustness against natural, out-of-distribution shifts that frequently occur in real-world environments, including snow, rain, fog, and brightness on SVHN, GTSRB, CURE-TSR, and ImageNet.
• ImageNet-c robustness. We show that our algorithms can significantly improve the robustness of classifiers trained on ImageNet and tested on ImageNet-c by as much as 30 percentage points.
• Robustness to simultaneous distributional shifts. We show that our methods are composable and can improve robustness to multiple simultaneous sources of natural variation. To evaluate this feature, we curate several new datasets, each of which has two simultaneous distributional shifts.
• Robustness to unseen domains. We show that models of natural variation can be reused on datasets that are entirely unseen during training to improve out-of-distribution generalization.

heading: 2 PERTURBATION-BASED ROBUSTNESS: APPROACHES AND LIMITATIONS

text:
In this paper, we consider a standard classification task in which training data (x, y) ⇠ D is distributed according to a joint distribution D over instances x 2 Rd and labels y 2 [k] := {0, 1, . . . , k}. In this setting, given a finite training sample drawn i.i.d. from D, the goal of the learning problem is to obtain a classifier fw parameterized by weights w 2 Rp such that fw can correctly predict the labels y corresponding to new instances x drawn i.i.d. from D. In practice, one can learn fw by approximately solving the non-convex empirical risk-minimization (ERM) problem argminw E[`(x, y;w)] where ` is a suitable loss-function. However, neural networks trained using ERM are known to be susceptible to adversarial attacks. This means that given a datum x with a corresponding label y, one can find another datum xadv such that x is close to xadv in a given Euclidean norm and xadv is predicted by the learned classifier as belonging to a different class c 6= y. If such a datum xadv exists, it is called an adversarial example. This is illustrated in Figure 1a; although these pandas look identical, they were classified differently in (Goodfellow et al., 2014b).
(a) Models take the form G(x, ), where is a nuisance parameter that describes how the output image x0 := G(x, ) is varied. (b) Input image x and corresponding generated images for a learned model of natural variation on ImageNet.
Figure 2: In this paper, we introduce models of natural variation to describe natural transformations.
The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data (Madry et al., 2017; Wong & Kolter, 2017). This can be formulated as follows:
argmin w E(x,y)⇠D h max 2 `(x+ , y;w) i
(1)
We can think of (1) as comprising two coupled optimization problems: an inner maximization problem in which we seek a challenging perturbation and an outer minimization problem in which we seek weights that lead to strong classification performance.
Limitations of perturbation-based robustness. Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations, there are significant limitations to adversarial training. Notably, DL is known to be fragile to many forms of natural variation, which cannot be described by small perturbations x 7! x+ . In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019b; Hosseini & Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018b; Karianakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation.

heading: 3 A NEW ROBUSTNESS PARADIGM: MODEL-BASED ROBUST DEEP LEARNING

text: Underlying the task of improving the robustness of neural networks against natural, out-ofdistribution data are two fundamental challenges. Firstly, unlike in the adversarial robustness community, in real-world, safety-critical environments, data can vary in unknown and highly nonlinear ways. Thus, the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments. Next, assuming a suitable model of natural variation, the second challenge is to formulate a training procedure that exploits this model toward improving robustness. In this section, we present novel solutions to each of these challenges.

heading: 3.1 MODELS OF NATURAL VARIATION

text:
In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define models of natural variation. Concretely, a model of natural variation G(x, ) is a map that describes how an input datum x can be naturally varied by a nuisance parameter resulting in a new datum x0 := G(x, ). Ideally, for a fixed datum x, varying the nuisance parameter should vary the severity of the natural conditions in the generated datum x0. An example of such a model is shown in Figure 2, where an image x on the left (in this case, in sunny weather) can be naturally varied by and consequently transformed into the image x0 on the right (in snowy weather). In the remainder of this subsection, we consider cases in which (1) a model G is known a priori, and (2) a model G is unknown and therefore must be learned offline from data. In this second case in which models of natural variation must be learned, we propose a method for obtaining these models.
Known models of natural variation. In many problems, a model G(x, ) is known a priori due to intrinsic geometric structure. For example, there is underlying structure that describes how data can
be rotated, translated, or scaled; models for rotating an image can be characterized by G(x, ) = R( )x where R( ) is a rotation matrix and 2 := [0, 2⇡). In prior work, this idea has been used to train classifiers to be robust to rotation and scaling (Engstrom et al., 2017; Kamath et al., 2020).
Learning models of natural variation from data. In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 2 in which a model G(x, ) takes an image x of a street sign in sunny weather and maps it to an image x0 := G(x, ) in snowy weather. Even though there is a relationship between the two images, obtaining a model G relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for learning the model G from data. To do so, we assume that we have access to two unpaired domains A and B that are drawn from a common distribution. Domain A contains the original data, such as the images with sunny weather, and domain B contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain A to corresponding images with different levels of natural variation captured by the images of domain B. In our experiments section, we rely on the MUNIT framework (Huang et al., 2018), which combines two autoencoders and two generative adversarial networks (Goodfellow et al., 2014a), to learn mappings between domains A and B. Furthermore, we note that many choices unpaired, unconditional image-to-image translation networks satisfy our criteria for G, and in future work we plan to investigate the efficacy of these architectures. In Appendix A, we describe parallel experiments that we carried out with two other architectural choices for G, and we fully characterize the MUNIT architecture used in our experiments.

heading: 3.2 MODEL-BASED ROBUST TRAINING FORMULATION

text:
The model-based robust training paradigm that we propose retains the basic elements of adversarial training described in Section 2. Our point of departure from the classical adversarial training formulation is in the choice of the so-called adversarial perturbation. In this paper, we assume that data can be transformed according to a model of natural variation G(x, ) by choosing different values of from a given nuisance space . The goal of the model-based approach is to train a classifier that achieves high accuracy both on a test set drawn i.i.d. from D and on more-challenging test data that has been subjected to the source of natural variation that G models. This perspective can be captured by the following robust optimization problem:
min w E(x,y)⇠D h max 2 `(G(x, ), y;w) i . (2)
In the inner maximization problem of this formulation, given an instance-label pair (x, y), we seek a vector ⇤ 2 that produces a corresponding instance x0 := G(x, ⇤) which gives rise to high loss values `(G(x, ⇤), y;w) under the current weight w. One can think of this vector ⇤ as characterizing the worst-case nuisance that can be generated by the model G(x, ⇤) for the original instance x. After solving this inner problem, we solve the outer minimization problem by finding weights w that minimize the risk against the challenging instance G(x, ⇤). By training the network to correctly classify this worst-case data, the goal is to become invariant to the model G(x, ) for any 2 .

heading: 4 MODEL-BASED TRAINING ALGORITHMS

text:
We now assume that we have access to a suitable model of natural variation G(x, ) and shift our attention toward exploiting G in the development of novel robust training algorithms. In the empirical version of (2), rather than assuming access to the full joint distribution D, we assume that we are given given a set of i.i.d. samples Dn := {(xj , yj)}nj=1 drawn from D. Thus we have:
w? 2 argmin w2Rp
nX
j=1
h max 2 ` (G (xj , ) , yj ;w) i . (3)
Note that when w parameterizes a neural network, (3) is a nonconvex-nonconcave min-max problem, which is difficult to solve exactly. Thus, we resort to approximate optimization techniques for solving (3). Specifically, we propose three algorithmic variants: Model-based Adversarial Training (MAT), Model-based Robust Training (MRT), and Model-based Data Augmentation (MDA). Pseudocode for MAT is given in Algorithm 1; pseudocode for MRT and MDA as well as further discussion of these algorithms is given in Appendix B. At a high level, each of these methods alternates between solving the outer minimization problem and the inner maximization problem. To this
Algorithm 1 Model-based Adversarial Training (MAT) Input: Data sample Dn = {(xj , yj)}nj=1, number of steps k, step size ↵ Output: Learned weight w
1: repeat 2: for minibatch Bm := {(x1, y1), (x2, y2), . . . , (xm, ym)} ⇢ Dn do 3: Initialize := ( 1, 2, . . . , m) (0q, 0q, . . . , 0q) 4: for k steps do 5: g r Pm j=1 `(G(xj , j), yj ;w) 6: ⇧ [ + ↵g] # ⇧ denotes projection onto the set 7: end for 8: g rw Pm j=1[`(G(xj , j), yj ;w) + · `(xj , yj ;w)]
9: w Update(g, w) # Update can be SGD, Adam, Adadelta, etc. 10: end for 11: until convergence
end, each of these algorithms uses SGD to solve the outer problem; the methods differ in how they seek solutions to the inner problem, and in what follows, we describe each of these procedures in more detail. In each algorithm, given a datum (x, y), the solution ? to the inner problem is used to create a new datum (G(x, ?), y) that is added to the training set before solving the outer problem.
Model-based Adversarial Training. In MAT, we seek an exact solution to the inner problem by performing k steps of gradient ascent in on the objective `(G(x, ), y;w). The resulting nuisance parameter ? is one that causes `(G(x, ?), y;w) to have high loss under the current weight w.
Model-based Robust Training. In MRT, we first randomly sample i 2 for i 2 [k]. We then select the i? 2 [k] such that `(G(x, i?), y;w) is maximized. In this way, rather than exactly solving the inner problem, MRT uses a sampling-based approach to finding challenging data G(x, i?).
Model-based Data Augmentation. In MDA, rather than explicitly trying to solve the inner problem, we seek a diversity of naturally-varying data rather than the “worst-case.” In this way, MDA samples i 2 for i 2 [k] and then appends {G(x, i), y}ki=1 to the training dataset.

heading: 5 EXPERIMENTS

text: We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1-5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3, we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4, we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5, we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods. Throughout these experiments, we use the notation “source (A!B)” to denote a distributional shift from domain A to domain B. For example, “contrast (low!high)” will denote a shift from low-contrast to high-contrast. Images from domains A and B for each of the shifts used in this paper are available in Appendix A. We note that our experiments contain domains with both natural and artificially-generated variation; details concerning how we extracted non-artificial variation can be found in Appendix D. Architecture and hyperparameter details are given in Appendix C.

heading: 5.1 OUT-OF-DISTRIBUTION ROBUSTNESS

text:
In many applications, one might have data corresponding to low levels of natural variation, such as a dusting of snow in images of street signs. However, it is often difficult to collect data corresponding to high levels of natural variation, such as images taken during a blizzard. In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of
natural variation. To do so, we use data from the CURE-TSR dataset (Temel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the “snow” subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1, we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR. We then train classifiers using MDA with labeled level 0 data. We also train classifiers using ERM and PGD using the labeled data from levels 0 and 2. We then test all classifiers on data from levels 3, 4, and 5. Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data.

heading: 5.2 MODEL-BASED ROBUSTNESS ON THE SHIFT FROM IMAGENET TO IMAGENET-C

text: To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks & Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019a).

heading: 5.3 ROBUSTNESS TO SIMULTANEOUS DISTRIBUTIONAL SHIFTS

text:
In practice, it is common to encounter multiple simultaneous distributional shifts. For example, in image classification, there may be shifts in both brightness and contrast; yet while there may be examples corresponding to shifts in either brightness or contrast in the training data, there may not be any examples of both shifts occurring simultaneously. To address this robustness challenge,
for each row of Table 3, we learn two models of natural variation G1 and G2 using unlabeled training data corresponding to two separate shifts, which map domains A1!B1 (e.g. low- to highbrightness) and A2!B2 (e.g. low- to high-contrast). We then compose these models to form a new model G(x, ) = G1(G2(x, ), ) which can be used to provide robustness against both shifts simultaneously. We then train classifiers on labeled data from A1\A2 and test on data from B1\B2. To create the data from B1 \ B2 for the ImageNet experiments, we apply pairs of transformations that were originally used to create the ImageNet-c datasets; more details are in Appendix D.

heading: 5.4 TRANSFERABILITY OF MODEL-BASED ROBUSTNESS

text: Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are entirely unseen while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation. For example, one might have access to two domains corresponding to the shift from images of European street signs taken during the day to images taken at night. However, one might wish to provide robustness against the same shift from daytime to nighttime on a new dataset of American street signs without access to any nighttime images in this new dataset. Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model G is learned on one dataset D1 and then applied on another D2; we improve robustness on unseen domains by up to 40 percentage points.

heading: 5.5 MODEL-BASED ROBUST DEEP LEARNING FOR UNSUPERVISED DOMAIN ADAPTATION

text:
While our approach does not require labeled data from domain B, when such data is available, it is of interest to evaluate how our approach compares to relevant methods such as domain adaptation. In Table 5, for each shift from domain A to B, we assume access to labeled data from domain A
and unlabeled data from domain B. In each row, we use unlabeled data from both domains to train a model of natural variation. We then train classifiers using our algorithms, as well with ERM and PGD, using data from domain A and test on data from the test set for domain B. Furthermore, we compare to ADDA, which is a well-known domain adaptation method (Tzeng et al., 2017). In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points. Note that while this is one of the most commonly studied settings in domain adaptation, it represents only one particular setting to which the MBRDL paradigm can be applied.

heading: 6 RELATED WORK

text:
Aside from the algorithms we introduced in Section 4, we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section. However, several lines of research have sought to address this problem in constrained settings or under highly restrictive assumptions. In the domain adaptation literature, various methods have been proposed which rely on the restrictive assumption that unlabeled data corresponding to a fixed distributional shift is available during training (Tzeng et al., 2017; Ajakan et al., 2014; Ganin & Lempitsky, 2015). Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training. Furthermore, several works have used generative models to create adversarial perturbations (Xiao et al., 2018a; Lee et al., 2017; Wang & Yu, 2019; Samangouei et al., 2018; Jalal et al., 2017) or perceptually-realistic images subject to relatively simple corruptions in specific application domains (Dunn et al., 2019; Song et al., 2018; Vandenhende et al., 2019; Arruda et al., 2019). On the other hand, our approach is broadly applicable to arbitrary and challenging sources of natural variation.
Two concurrent works formulate robust training procedures assuming that data is corrupted according to a fixed generative architecture. Gowal et al. (2020) exploit properties specific to the StyleGAN architecture to formulate a training algorithm that provides robustness against color-based shifts on MNIST and CelebA. In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets. Wong & Kolter (2020) use conditional VAEs to learn perturbation sets corresponding to simple corruptions from pairs of images. In our framework we improve robustness against more challenging, natural shifts by learning from unpaired datasets and we do not rely on class-conditioning to generate realistic images.

heading: 7 CONCLUSION

text: In this paper, we proposed a novel model-based robust training paradigm for deep learning that provides robustness with respect to models of natural variation. Our notion of robustness offers a departure from adversarial training with respect to norm-bounded data perturbations. In our experiments, we show that our paradigm can provide significant out-of-distribution robustness on many challenging distributional shifts. Furthermore, our paradigm can provide robustness against multiple simultaneous distribution shifts and on domains that are entirely unseen while training the model, and shows significant out-of-distribution robustness as datasets become more challenging.

title: MODEL-BASED ROBUST DEEP LEARNING: GENERAL-