summaries: The work presents a causal perspective of adversarial attacks on image-based machine learning models by studying a causal graph of the adversarial data creation process and highlighting how such a process makes the learned models vulnerable. It argues that the main reason for adversarial vulnerability is the reliance of models on spurious correlations between labels and style. Accordingly, it proposes a method to learn models for which the conditional distribution of label given style and image does not vary much when attacked. Empirically, the method is shown to be more robust than two baselines on three datasets. This paper presents a causal perspective on addressing adversarially vulnerability. It first constructs a causal graph, which then inspires the design of the distribution alignment method for reducing the gap between adversarial and natural data. Extensive experiments on CIFAR10, CIFAR100, and MNIST demonstrate the robustness of the proposed method against various attack methods. The paper shows a causal perspective to the adversarial robustness problem. It creates a graph over content and style variable sets. It identifies the spurious correlation between style and label as the main reason for adversarial examples, and then proposes a method to remove it from the trained model. Experiments on three datasets show that the proposed method is better than two baselines.  This paper proposes a causal graph to model the generation process of adversarial attacks. Based on the proposed causal graph, the authors identify the origin of adversarial vulnerability as the spurious correlation between style variable and class label. Under the adversarial distribution, such spurious correlation can be maliciously used to mislead a victim model. In this light, the authors propose a method to align the adversarial distribution and the natural distribution to prevent a model from learning spurious correlation. The proposed method is empirically validated on prevailing datasets under several attacks.
abstractText: The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. As causal reasoning has an instinct for modeling distribution change, it is essential to incorporate causality into analyzing this specific type of distribution change induced by adversarial attacks. However, causal formulations of the intuition of adversarial attacks and the development of robust DNNs are still lacking in the literature. To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and define the adversarial distribution to formalize the intuition of adversarial attacks. From the causal perspective, we study the distinction between the natural and adversarial distribution and conclude that the origin of adversarial vulnerability is the focus of models on spurious correlations. Inspired by the causal understanding, we propose the adversarial distribution alignment method to eliminate the difference between natural and adversarial distributions by considering spurious correlations. Extensive experiments demonstrate the efficacy of the proposed method. Our work is the first attempt towards using causality to understand and mitigate adversarial vulnerability.
references.authors: Anish Athalye Nicholas Carlini David Wagner
references.title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
references.venue: In International Conference on Machine Learning,
references.authors: Yoshua Bengio Aaron Courville Pascal Vincent
references.title: Representation learning: A review and new perspectives
references.venue: IEEE transactions on pattern analysis and machine intelligence,
references.authors: Battista Biggio Igino Corona Davide Maiorca Blaine Nelson Nedim Šrndić Pavel Laskov Giorgio Giacinto Fabio Roli
references.title: Evasion attacks against machine learning at test time
references.venue: In Joint European conference on machine learning and knowledge discovery in databases,
references.authors: Peter Bühlmann
references.title: Invariance, causality and robustness
references.venue: Statistical Science,
references.authors: Nicholas Carlini David Wagner
references.title: Towards evaluating the robustness of neural networks
references.venue: In 2017 ieee symposium on security and privacy (sp),
references.authors: Yair Carmon Aditi Raghunathan Ludwig Schmidt Percy Liang John C Duchi
references.title: Unlabeled data improves adversarial robustness
references.venue: arXiv preprint arXiv:1905.13736,
references.authors: Ting Chen Simon Kornblith Mohammad Norouzi Geoffrey Hinton
references.title: A simple framework for contrastive learning of visual representations
references.venue: In International conference on machine learning,
references.authors: Juan Correa Elias Bareinboim
references.title: A calculus for stochastic interventions: Causal effect identification and surrogate experiments
references.venue: In Proceedings of the AAAI Conference on Artificial Intelligence,
references.authors: Francesco Croce Matthias Hein
references.title: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
references.venue: In International conference on machine learning,
references.authors: Yinpeng Dong Fangzhou Liao Tianyu Pang Hang Su Jun Zhu Xiaolin Hu Jianguo Li
references.title: Boosting adversarial attacks with momentum
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Frederick Eberhardt Richard Scheines
references.title: Interventions and causal inference
references.venue: Philosophy of science,
references.authors: Yarin Gal Zoubin Ghahramani
references.title: Dropout as a bayesian approximation: Representing model uncertainty in deep learning
references.venue: In international conference on machine learning,
references.authors: Yaroslav Ganin Evgeniya Ustinova Hana Ajakan Pascal Germain Hugo Larochelle François Laviolette Mario Marchand Victor Lempitsky
references.title: Domain-adversarial training of neural networks. The journal of machine learning
references.authors: Ruize Gao Feng Liu Jingfeng Zhang Bo Han Tongliang Liu Gang Niu Masashi Sugiyama
references.title: Maximum mean discrepancy test is aware of adversarial attacks
references.venue: In International Conference on Machine Learning,
references.authors: Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix A Wichmann Wieland Brendel
references.title: Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness
references.venue: In International Conference on Learning Representations,
references.authors: Ian J Goodfellow Jonathon Shlens Christian Szegedy
references.title: Explaining and harnessing adversarial examples
references.venue: arXiv preprint arXiv:1412.6572,
references.authors: Alison Gopnik Clark Glymour David M Sobel Laura E Schulz Tamar Kushnir David Danks
references.title: A theory of causal learning in children: causal maps and bayes nets
references.venue: Psychological review,
references.authors: Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
references.title: Deep residual learning for image recognition
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross Girshick
references.title: Momentum contrast for unsupervised visual representation learning
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Christina Heinze-Deml Nicolai Meinshausen
references.title: Conditional variance penalties and domain shift robustness
references.venue: Machine Learning,
references.authors: Jan Hendrik Metzen Mummadi Chaithanya Kumar Thomas Brox Volker Fischer
references.title: Universal adversarial perturbations against semantic image segmentation
references.venue: In Proceedings of the IEEE International Conference on Computer Vision,
references.authors: Geoffrey Hinton Oriol Vinyals Jeff Dean
references.title: Distilling the knowledge in a neural network
references.venue: arXiv preprint arXiv:1503.02531,
references.authors: Andrew Ilyas Shibani Santurkar Logan Engstrom Brandon Tran Aleksander Madry
references.title: Adversarial examples are not bugs, they are features
references.venue: Advances in neural information processing systems,
references.authors: Alex Kendall Yarin Gal
references.title: What uncertainties do we need in bayesian deep learning for computer vision
references.venue: In Proceedings of the 31st International Conference on Neural Information Processing Systems,
references.authors: Niki Kilbertus Giambattista Parascandolo Bernhard Schölkopf
references.title: Generalization in anti-causal learning
references.venue: arXiv preprint arXiv:1812.00524,
references.authors: Alex Krizhevsky Geoffrey Hinton
references.title: Learning multiple layers of features from tiny images
references.authors: Alex Krizhevsky Ilya Sutskever Geoffrey E Hinton
references.title: Imagenet classification with deep convolutional neural networks
references.venue: Advances in neural information processing systems,
references.authors: Yann LeCun Léon Bottou Yoshua Bengio Patrick Haffner
references.title: Gradient-based learning applied to document recognition
references.venue: Proceedings of the IEEE,
references.authors: Xingjun Ma Bo Li Yisen Wang Sarah M Erfani Sudanthi Wijewickrema Grant Schoenebeck Dawn Song Michael E Houle James Bailey
references.title: Characterizing adversarial subspaces using local intrinsic dimensionality
references.venue: In ICLR,
references.authors: Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu
references.title: Towards deep learning models resistant to adversarial attacks
references.venue: arXiv preprint arXiv:1706.06083,
references.authors: Jovana Mitrovic Brian McWilliams Jacob C Walker Lars Holger Buesing Charles Blundell
references.title: Representation learning via invariant causal mechanisms
references.venue: In International Conference on Learning Representations,
references.authors: Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi Pascal Frossard
references.title: Deepfool: a simple and accurate method to fool deep neural networks
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi Omar Fawzi Pascal Frossard
references.title: Universal adversarial perturbations
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Amir Najafi Shin-ichi Maeda Masanori Koyama Takeru Miyato
references.title: Robustness to adversarial perturbations in learning from incomplete data
references.authors: Tianyu Pang Xiao Yang Yinpeng Dong Kun Xu Jun Zhu Hang Su
references.title: Boosting adversarial training with hypersphere embedding
references.venue: arXiv preprint arXiv:2002.08619,
references.authors: Nicolas Papernot Patrick McDaniel Ian Goodfellow Somesh Jha Z Berkay Celik Ananthram Swami
references.title: Practical black-box attacks against machine learning
references.venue: In Proceedings of the 2017 ACM on Asia conference on computer and communications security,
references.authors: Jonas Peters Dominik Janzing Bernhard Schölkopf
references.title: Elements of causal inference: foundations and learning algorithms
references.authors: Aditi Raghunathan Jacob Steinhardt Percy Liang
references.title: Certified defenses against adversarial examples
references.venue: arXiv preprint arXiv:1801.09344,
references.authors: Nasim Rahaman Aristide Baratin Devansh Arpit Felix Draxler Min Lin Fred Hamprecht Yoshua Bengio Aaron Courville
references.title: On the spectral bias of neural networks
references.venue: In International Conference on Machine Learning,
references.authors: Leslie Rice Eric Wong Zico Kolter
references.title: Overfitting in adversarially robust deep learning
references.venue: In International Conference on Machine Learning,
references.authors: Shiori Sagawa Pang Wei Koh Tatsunori B Hashimoto Percy Liang
references.title: Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization
references.authors: Axel Sauer Andreas Geiger
references.title: Counterfactual generative networks
references.venue: arXiv preprint arXiv:2101.06046,
references.authors: Bernhard Schölkopf Francesco Locatello Stefan Bauer Nan Rosemary Ke Nal Kalchbrenner Anirudh Goyal Yoshua Bengio
references.title: Toward causal representation learning
references.venue: Proceedings of the IEEE,
references.authors: Ali Shafahi Mahyar Najibi Amin Ghiasi Zheng Xu John Dickerson Christoph Studer Larry S Davis Gavin Taylor Tom Goldstein
references.title: Adversarial training for free
references.authors: Karen Simonyan Andrew Zisserman
references.title: Very deep convolutional networks for large-scale image recognition
references.venue: arXiv preprint arXiv:1409.1556,
references.authors: Sahil Singla Soheil Feizi
references.title: Second-order provable defenses against adversarial attacks
references.venue: In International Conference on Machine Learning,
references.authors: Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian J. Goodfellow Rob Fergus
references.title: Intriguing properties of neural networks
references.venue: In 2nd International Conference on Learning Representations,
references.authors: Christian Szegedy Wei Liu Yangqing Jia Pierre Sermanet Scott Reed Dragomir Anguelov Dumitru Erhan Vincent Vanhoucke Andrew Rabinovich
references.title: Going deeper with convolutions
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Kaihua Tang Jianqiang Huang Hanwang Zhang
references.title: Long-tailed classification by keeping the good and removing the bad momentum causal effect
references.venue: Advances in Neural Information Processing Systems,
references.authors: Takeshi Teshima Issei Sato Masashi Sugiyama
references.title: Few-shot domain adaptation by causal mechanism transfer
references.venue: In International Conference on Machine Learning,
references.authors: Florian Tramer Nicholas Carlini Wieland Brendel Aleksander Madry
references.title: On adaptive attacks to adversarial example defenses
references.venue: Advances in Neural Information Processing Systems,
references.authors: Chun-Chen Tu Paishun Ting Pin-Yu Chen Sijia Liu Huan Zhang Jinfeng Yi Cho-Jui Hsieh Shin-Ming Cheng
references.title: Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks
references.venue: In Proceedings of the AAAI Conference on Artificial Intelligence,
references.authors: Cumhur Erkan Tuncali Georgios Fainekos Hisahiro Ito James Kapinski
references.title: Simulation-based adversarial test generation for autonomous vehicles with machine learning components
references.venue: IEEE Intelligent Vehicles Symposium (IV),
references.authors: Yisen Wang Difan Zou Jinfeng Yi James Bailey Xingjun Ma Quanquan Gu
references.title: Improving adversarial robustness requires revisiting misclassified examples
references.venue: In International Conference on Learning Representations,
references.authors: Eric Wong Zico Kolter
references.title: Provable defenses against adversarial examples via the convex outer adversarial polytope
references.venue: In International Conference on Machine Learning,
references.authors: Eric Wong Leslie Rice J Zico Kolter
references.title: Fast is better than free: Revisiting adversarial training
references.venue: arXiv preprint arXiv:2001.03994,
references.authors: Dongxian Wu Shu-Tao Xia Yisen Wang
references.title: Adversarial weight perturbation helps robust generalization
references.venue: arXiv preprint arXiv:2004.05884,
references.authors: Cihang Xie Yuxin Wu Laurens van der Maaten Alan L Yuille Kaiming He
references.title: Feature denoising for improving adversarial robustness
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Yuzhe Yang Guo Zhang Dina Katabi Zhi Xu
references.title: Me-net: Towards effective adversarial robustness with matrix estimation
references.venue: In International Conference on Machine Learning,
references.authors: Sergey Zagoruyko Nikos Komodakis
references.title: Wide residual networks. In British Machine Vision Conference 2016
references.venue: British Machine Vision Association,
references.authors: Cheng Zhang Kun Zhang Yingzhen Li
references.title: A causal view on robustness of neural networks
references.venue: Advances in Neural Information Processing Systems,
references.authors: Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals
references.title: Understanding deep learning (still) requires rethinking generalization
references.venue: Communications of the ACM,
references.authors: Hongyang Zhang Yaodong Yu Jiantao Jiao Eric Xing Laurent El Ghaoui Michael Jordan
references.title: Theoretically principled trade-off between robustness and accuracy
references.venue: In International Conference on Machine Learning,
references.authors: Jingfeng Zhang Jianing Zhu Gang Niu Bo Han Masashi Sugiyama Mohan Kankanhalli
references.title: Geometry-aware instance-reweighted adversarial training
references.venue: arXiv preprint arXiv:2010.01736,
references.authors: Kun Zhang Bernhard Schölkopf Krikamol Muandet Zhikun Wang
references.title: Domain adaptation under target and conditional shift
references.venue: In International Conference on Machine Learning,
references.authors: Yonggang Zhang Xinmei Tian Ya Li Xinchao Wang Dacheng Tao
references.title: Principal component adversarial example
references.venue: IEEE Transactions on Image Processing,
references.authors: Zhang
references.title: Following (Zhang et al., 2019), the network is trained using SGD
references.authors: Xie et al Yang
references.title: 2019), especially the adversarial training method (Goodfellow
sections.text: The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. As causal reasoning has an instinct for modeling distribution change, it is essential to incorporate causality into analyzing this specific type of distribution change induced by adversarial attacks. However, causal formulations of the intuition of adversarial attacks and the development of robust DNNs are still lacking in the literature. To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and define the adversarial distribution to formalize the intuition of adversarial attacks. From the causal perspective, we study the distinction between the natural and adversarial distribution and conclude that the origin of adversarial vulnerability is the focus of models on spurious correlations. Inspired by the causal understanding, we propose the adversarial distribution alignment method to eliminate the difference between natural and adversarial distributions by considering spurious correlations. Extensive experiments demonstrate the efficacy of the proposed method. Our work is the first attempt towards using causality to understand and mitigate adversarial vulnerability.
sections.heading: 1 INTRODUCTION
sections.text: The seminal work (Szegedy et al., 2014; Biggio et al., 2013) shows that DNNs are vulnerable to adversarial examples, which consist of malicious perturbations imperceptible to humans yet fooling state-of-the-art models (Krizhevsky et al., 2012; Szegedy et al., 2015; Simonyan & Zisserman, 2014; He et al., 2016). The lack of robustness hinders the applications of DNNs to some safetycritical areas such as automatic driving (Tuncali et al., 2018) and healthcare (Finlayson et al., 2019). Therefore, mitigating the adversarial vulnerability is critical to the further development of DNNs.
  Human cognitive systems are immune to the distribution change induced by adversarial attacks because humans are more sensitive to causal relations than statistical associations (Gopnik et al., 2004). Using causal language, causal reasoning can identify causal relation and ignore nuisance factors, i.e., not the cause of labels, by intervention (Pearl, 2009; Peters et al., 2017). As adversarial perturbations are usually imperceptible and make no impact on human decisions (Szegedy et al., 2014; Goodfellow et al., 2014), it is reasonable to assume that the difference between natural and adversarial examples comes from nuisance factors. This is because if task-relevant factors of some samples are changed, these samples will make both humans and DNNs change their decisions. From a causal viewpoint (Zhang et al., 2020a), adversarial attacks can be regarded as a specific type of distribution change resulting from the intervention on the natural data distribution. In summary, causal reasoning has the instinct for analyzing the effect of the intervention caused by adversarial attacks, so it is essential to leverage causality to understand and mitigate the adversarial vulnerability.
  However, there are two significant problems to overcome before using causality to understand and mitigate the adversarial vulnerability. Firstly, constructing a causal graph is arguably the fundamental premise for causal reasoning (Pearl, 2009; Peters et al., 2017), but how to construct causal graphs in the context of adversarial attacks is still lacking in the literature. Secondly, using causal language to formalize the intuition of adversarial attacks is the key to connect causality and adversarial vulnerability, but it also remains to be solved. These two problems are fundamental obstacles, which prevent us from employing causality to contribute to adversarial learning.
  To address these challenges, we first construct a causal graph to model the perceived data generation process where nuisance factors are considered. The constructed causal graph can entail a specific intervention distribution, i.e., the adversarial distribution. Moreover, the causal graph immediately shows that, given inputs, labels are statistically correlated with nuisance factors, which have no cause-effect to labels. The spurious correlation implies that if DNNs fit the conditional association between labels and nuisance factors, their performance on different conditional associations between labels and nuisance factors will change accordingly. Through investigating the distinction between these two distributions induced by nuisance factors, we conclude that adversarial distributions can be obtained by exploiting conditional associations between labels and nuisance factors, where the conditional association of adversarial distributions is drastically different from that of natural distributions. Namely, the origin of adversarial vulnerability is the focus of DNNs on the spurious correlation between labels and nuisance factors.
  According to the causal perspective, an adversarial distribution is crafted by exploiting a specific conditional association between labels and nuisance factors, this association is drastically different from that of natural distributions. Intuitively, eliminating the difference in such conditional associations between natural and adversarial distributions can promote the performance of models on adversarial distributions. Thus, we propose an adversarial distribution alignment method to eliminate the difference between these two distributions. Surprisingly, we find that the proposed method shares the same spirits to existing adversarial training (Goodfellow et al., 2014) variants, i.e., Madry (Madry et al., 2017) and TRADES (Zhang et al., 2019). We validate the efficacy of the proposed method on MNIST, CIFAR10, and CIFAR100 (Krizhevsky et al., 2009) datasets under various adversarial attacks such as FGSM (Goodfellow et al., 2014), PGD (Madry et al., 2017), CW attack (Carlini & Wagner, 2017), and AutoAttack (Croce & Hein, 2020). Extensive experiments demonstrate that the proposed method can improve the adversarial robustness significantly.
  Our main contributions are:
  • We provide a causal perspective to understand and mitigate the adversarial vulnerability, which is the first attempt towards using causality to contribute adversarial learning.
  • To leverage causality to contribute adversarial learning, we solve two fundamental problems. Specifically, we construct a causal graph to model the adversarial data generation process and define the adversarial distribution to formalize adversarial attacks.
  • A defense method called adversarial distribution alignment is proposed to reduce adversarial vulnerability by eliminating the difference between adversarial distribution and natural distribution. Extensive experiments demonstrate that the proposed method can significantly improve the adversarial robustness.
sections.heading: 2 A CAUSAL VIEW ON ADVERSARIAL DATA GENERATION
sections.text: The ability of humans to perform causal reasoning is arguably an essential factor that makes human learning different from deep learning (Schölkopf et al., 2021; Zhang et al., 2020a; Gopnik et al., 2004). The superiority of causal reasoning endows humans with the ability to identify causal relations and ignore nuisance factors that are not relevant to the task. In contrast, DNNs are usually trained to fit the perceived information overlooking the ability to distinguish causal relations and statistical associations. This shortcut solution could lead to overfitting to these nuisance factors, which would further result in the sensitivity of DNNs to such factors. Therefore, we propose incorporating causal reasoning to mitigate the sensitivity of DNNs to these nuisance factors.
  Before using causal reasoning to analyze adversarial vulnerability, we need to construct a causal graph, as causal graphs are the key for formalizing causal reasoning (Peters et al., 2017). In the context of adversarial learning, we desire the causal graph by which both the natural and the adversarial distributions can be generated. In addition, the graph is required to reflect the impact of nuisance factors on these two distributions, so that we can investigate the difference in nuisance factors between these two distributions. Consequently, we can formally establish the connection between nuisance factors and adversarial vulnerability. Therefore, we propose constructing a causal graph to model the adversarial generation process where the nuisance factors are considered. One approach is to use causal structure learning to infer causal graphs (Pearl, 2009; Peters et al., 2017), but it is challenging to apply this kind of approach to high-dimensional data. Using external knowl-
  edge to construct causal graphs is another approach (Zhang et al., 2013; Tang et al., 2020; Schölkopf et al., 2021). As automatically learning a precise causal graph is out of scope for this work, external human knowledge about the data generation process is employed to construct the causal graph.
  Specifically, we construct a causal graph G to formalize the image data generation process using the following knowledge for analyzing adversarial vulnerability. As there might be a number of different causes of natural data X , we propose to divide all the causes into two categories for simplicity. We group content-related causes into one category, called content variable C. The rest causes, i.e., nuisance factors, are grouped into another category, called style variable S, which is contentindependent, i.e., S ⫫ C. This implies that C → X ← S. It is noteworthy that, in this paper, we assume that only the content variable is relevant for the task we care about, i.e., C → Y . Perceived data X̃ are usually composed of perturbations E and natural data X . When the perturbation E is designed carefully to fool DNNs, E should be a compound result of X (the object to be perturbed), Y (the reference for the perturbation), and θ (the targets affected by the perturbation), e.g., white-box attacks (Goodfellow et al., 2014; Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016), which means that (X,Y,θ) → E 1. Leverage all this background knowledge, we obtain the causal graph G formalizing the perturbed data generation process, depicted in Fig. 1. Based on the causal graph, we can define valid interventions (Pearl, 2009; Schölkopf et al., 2021) and the corresponding intervention distributions. Defining valid interventions is equivalent to determining which variables or mechanisms in the causal graph can be intervened. In this work, we consider both hard and soft interventions (Eberhardt & Scheines, 2007; Correa & Bareinboim, 2020) on the perturbation variable E. Specifically, we can use a structural causal model to represent the generating mechanism of E: E ≔M(X,Y,θ, UE), (1) where the exogenous variable UE stands for other indeterminacies, e.g., random start noise used in PGD attack (Madry et al., 2017).
  By intervening E in different ways, we can obtain different intervention distributions, which could be either natural or adversarial, over the observed variables. For example, if we do a hard intervention on E, i.e., do(E = 0) in the graph, the generated data distribution corresponds to the natural distribution P (X,Y ). Different perturbations can be obtained by performing soft interventions on E, i.e., modifying M. In the context of adversarial attacks, adversaries aim to maximize a certain objective function `(⋅) to mislead a target model h (X;θ) by searching for the worst perturbation for each instance (Goodfellow et al., 2014; Carlini & Wagner, 2017; Dong et al., 2018; Madry et al., 2017), where θ stands for parameters of the target model. To formalize the intuition of adversarial attacks, we can first search for the adversarial perturbation Eadv , and then use a function to approximate the mechanism for generating Eadv . The adversarial perturbation Eadv can be obtained by maximizing `(⋅):
  Eadv = arg max E ′∈B
  `(h(X + E ′;θ), Y ). (2)
  1These three causes are not indispensable. Deleting θ gives black-box attacks (Papernot et al., 2017; Dong et al., 2018) that only use (X,Y ) to perform attacks. Deleting X leads to universal adversarial attacks (Moosavi-Dezfooli et al., 2017; Hendrik Metzen et al., 2017), which assume that one adversarial perturbation is sufficient to fool DNNs. Recent works show that a specific transform to X effectively evaluates the sensitivity of DNNs within a small neighborhood (Rahaman et al., 2019; Zhang et al., 2020c), which corresponds to deleting (Y,θ) from the cause set.
  where B is a set of valid perturbations, and the adversarial perturbation Eadv is the result of the mechanism Madv , i.e.,Eadv ≔Madv(X,Y,θ, UE). The intervention distribution that corresponds to the adversarial mechanism Madv is defined as the adversarial distribution Pθ(X̃, Y ), where the subscript θ indicates that Pθ(X̃, Y ) is crafted to attack the target model h (X;θ) with parameter θ.
sections.heading: 3 METHOD
sections.text: To understand the adversarial vulnerability, we study the difference between the natural and adversarial distributions and conclude that conditional associations between labels and style factors play a crucial role in adversarial vulnerability. Inspired by the conclusion, we propose a method to eliminate the difference in such conditional associations to mitigate adversarial vulnerability.
sections.heading: 3.1 ORIGIN OF ADVERSARIAL VULNERABILITY
sections.text: Inspired by the causal graph, we derive a causal understanding of adversarial vulnerability. According to the causal graph G depicted in Fig. 1, there is a path, S → X ← C → Y , from the style variable S to the label Y when X is given 2, which leads to the correlation between labels and style variables. The spurious correlation implies that DNNs can perform well on the training set by fitting the statistical association between Y and S, even though the genuine content information is dropped. Moreover, if the conditional association between Y and S on the (natural) test set is similar to that on the training set, fitting these spurious correlations will also perform well on the test set. This is consistent with the recent work (Ilyas et al., 2019), which shows that training DNNs with incorrectly labeled data, i.e., the genuine content information is not utilized for training, yields good accuracy on the (natural) test set. However, suppose DNNs learn such spurious correlation. In that case, their performance will change with the conditional association between Y and S. Consequently, fitting such spurious correlation gives adversaries a chance to fool DNNs. Therefore, the conditional association between labels and style variables is a key to understand adversarial vulnerability.
  To identify the origin of adversarial vulnerability from the spurious correlation perspective, we study the distinction between the adversarial distribution and the natural distribution. To look at what makes the adversarial distribution different from the natural distribution, we expand the natural P (Y ∣X) and the adversarial distribution Pθ(Y ∣X̃)
  P (Y ∣X) = ∑ s∈S P (s∣X)P (Y ∣X, s), Pθ(Y ∣X̃) = ∑ s∈S Pθ(s∣X̃)Pθ(Y ∣X̃, s), (3)
  where we assume the set of valid styles S is discrete, and s stands for a certain style. We can see that the difference between P (Y ∣X) and Pθ(Y ∣X̃) results from two terms, i.e., Pθ(s∣X̃) and Pθ(Y ∣X̃, s). Pθ(s∣X̃) represents the change of style information, e.g., textures (Geirhos et al., 2018), transformations (Chen et al., 2020; Mitrovic et al., 2020; He et al., 2020), and domain shifts (Ganin et al., 2016). It is shown that changing image textures has a significant impact on predictions of DNNs (Geirhos et al., 2018), but such drastic style changes will hardly appear in the context of adversarial attacks as perturbations are required to be imperceptible 3. Although modifying style variables S is not allowed, adversaries can exploit the conditional association between Y and S, i.e., Pθ(Y ∣X̃, s), via injecting a specific perturbation to generate adversarial distributions. Specifically, under an appropriate soft intervention, i.e., Madv , the conditional distribution Pθ(Y ∣X̃, s) can be drastically different from P (Y ∣X, s). Namely, the property of adversarial distribution essentially results from the drastic difference in the statistical association between labels and style variables. Intuitively, if DNNs fit the conditional associations between Y and S, their performance will change with the spurious correlation. Consequently, a drastic difference between the conditional association between Y and S will cause significant performance degradation of DNNs.
  Hence, the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables. This conclusion provides a causal perspective for the empirical
  2Here, we use the underlined variable X to represent that the variable X is given. 3In this work, we focus on imperceptible adversarial examples, i.e., the widely studied `p-norm bounded
  adversarial examples, where drastic style changes hardly appear.
  observation, i.e. some features are useful but not robust (Ilyas et al., 2019), so adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon.
sections.heading: 3.2 THE ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD
sections.text: In light of the causal understanding of the adversarial vulnerability, improving adversarial robustness requires forcing DNNs to fit the causal relations rather than merely the statistical associations. However, only the perceived data can be observed in practice, and the supervised information of content variables is usually unavailable. Therefore, an approach that can avoid reliance on such supervised information is required.
  To develop such an approach, we revisit the difference between the adversarial and natural distributions. Intuitively, if the difference between the natural and the adversarial distribution is negligible, the adversarial vulnerability of DNNs can be mitigated, as DNNs perform well on the natural distribution (Krizhevsky et al., 2012; He et al., 2016; Zhang et al., 2021). Consequently, a straightforward solution for improving adversarial robustness is to align these two distributions. The aforementioned analysis shows that the property of adversarial distributions to fool DNNs comes from specific conditional associations between labels Y and style variables S, i.e., Pθ(Y ∣X̃, s) and P (Y ∣X, s). Inspired by the conclusion, we propose an adversarial distribution alignment method to eliminate the difference between the natural and adversarial distributions. Specifically, we regard the natural distribution P (Y ∣X, s) as an anchor, then align the adversarial distribution Pθ(Y ∣X̃, s) with the anchor such that the difference between these two distributions is negligible. In addition, the relationship between Y and X should also be the same on both the adversarial and natural distributions. Concretely, we operationalize the intuition of the adversarial distribution alignment method by:
  min θ
  d (P (Y ∣X) , Pθ (Y ∣X̃)) + λEsd (P (Y ∣X, s) , Pθ (Y ∣X̃, s)) . (4)
  Here d(⋅) is a metric reflecting the divergence between two distributions, and λ > 0 presents the weighting of the misalignment penalty. In addition, we assume X and X̃ have the same support set because adversarial perturbations are usually bounded. Solving the adversarial distribution alignment objective is equivalent to search a classifier h(X̃;θ) such that the adversarial distribution of the classifier is similar to the natural distribution. Benefiting from the consistency of these two distributions, classifier h(X̃;θ) can perform well on both the natural and adversarial distributions. In practice, the adversarial distribution often cannot be obtained analytically, so we relax the distribution divergence in Eq. 4 to the sum of two divergences , see Appendix for details A. For the divergence between P (Y ∣X) and Pθ(Y ∣X̃), we have:
  KL (Pθ (Y ∣X̃) , Qθ (Y ∣X̃)) + γKL (P (Y ∣X) , Qθ (Y ∣X)) =E(X̃,Y )∼Pθ(X̃,Y )CE (h (X̃;θ) , Y ) + γE(X,Y )∼P (X,Y )CE (h (X;θ) , Y ) ≈E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y ) , (5)
  where KL is the Kullback-Leibler divergence, CE is the cross-entropy loss, Qθ(Y ∣X) is the conditional distribution specified by the classifier h (X;θ), and γ is a tunable hyperparameter. Because adversarial examples are usually generated by adding perturbation to their corresponding natural samples rather than sampled from the adversarial distribution independently, we use X + Eadv to approximate examples independently sampled from the adversarial distribution, whereX is sampled from the natural distribution independently.
  Similar to Eq. 5, the divergence between P (Y ∣X̃, s) and Pθ(Y ∣X̃, s) can be approximated by: E(X,Y )∼P (X,Y )CE (g (s (X + Eadv) ;Wg) , Y ) + β (g (s (X) ;Wg) , Y ) , (6)
  where g (s (X) ;Wg) is a function used for modeling the statistically conditional association between labels and style variables, s (X) stands for the integrated representation of X and s, and β is a tunable hyperparameter. Thus, the overall objective of the proposed adversarial distribution alignment method can be expressed as
  min θ,Wg
  E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y )
  + λ (EsCE (g (s (X + Eadv) ;Wg) , Y ) + βCE (g (s (X) ;Wg) , Y )) . (7)
  To make sure that introducing g can benefit learning h, it is necessary to design an approach to connect model g and model h. In this paper, we connect g and h by representation sharing. Interestingly, according to Eq. 4 and Eq. 7, if we omit the spurious correlation between labels and style variables, i.e., λ = 0, and set γ = 0, Eq. 7 then becomes the objective function introduced by Madry (Madry et al., 2017). This suggests that the proposed adversarial distribution alignment method is consistent with the seminal variant Madry (Madry et al., 2017) of adversarial training.
sections.heading: 3.3 REALIZATION OF ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD
sections.text: According to Eq. 7, realizing the proposed adversarial distribution alignment method requires predicting labels with all style variables, i.e., modeling the statistically conditional association between Y and S. However, there are two major obstacles preventing us from calculating the last term of Eq. 7. Specifically, a) the number of all possible styles s is infinite, so the cost of calculating the expectation in Eq. 7 can grow to infinity; b) the representation of the integrated representation of X and s, i.e., s (X), used for predicting labels is unknown. To calculate the expectation in Eq. 7, we have to make some assumptions to approximate the distribution of the style variable, as the true distribution is unknown. Following previous work (Gal & Ghahramani, 2016; Kendall & Gal, 2017), we assume a Gaussian distribution to approximate the unknown distribution. Specifically, the representation s (X) is estimated by ŝ (X) sampled from a Gaussian distribution, i.e., ŝ (X) ∼ N (µ (X) ,Σ), where µ (X) is employed to model the Gaussian distribution’s mean, and Σ is the covariance matrix. Because little prior knowledge about the covariance matrix is observed, we simply regard the covariance matrix Σ as an identity matrix, i.e., ŝ (X) ∼ N (µ (X) , σ2I). Then, the expectation can be approximated by
  EsCE (g (s (X) ;Wg) , Y ) ≈ Eŝ(X)∼N (µ(X),σ2I)CE (g (ŝ (X) ;Wg) , Y ) . (8) Thanks to the Gaussian distribution assumption, we can derive an upper bound of the expectation:
  Eŝ(X)∼N (µ(X),σ2I)CE (g (ŝ (X) ;Wg) , Y ) ≤ CE (g (µ (X) ;Wg) , Y ) , (9)
  where the probability of the ith category of g (µ (X) ;Wg) is (see Appendix B for details)
  P (Y = i∣g (µ (X) ;Wg)) = e W ⊤ g,iµ(X)
  ∑j eW ⊤ g,jµ(X)+σ
  2 2 (Wg,j−Wg,i)⊤(Wg,j−Wg,i)
  . (10)
  Here, we simply set g to a linear function, i.e., Wg is a linear mapping. Instead of calculating all styles ŝ (X), the conclusion of Eq. 9 and Eq. 10 shows that only the mean and variance of ŝ (X) are required to calculate the expectation in Eq. 7.
  In light of Eq. 9 and Eq. 10, the challenge of learning the representation of s (X) boils down to estimating the mean of ŝ (X), i.e., no need to estimate every style representation explicitly. In addition, the variance can be treated as a hyperparameter. To estimate the mean, we draw inspiration from the proposed causal graph. According to the causal graph G, the content and style variables are statistically independent, i.e., C ⫫ S. Thus, the estimated content ĉ (X) and style ŝ (X) are desired to be independent, i.e., ĉ (X) ⫫ ŝ (X). The underlying intuition is that we aim to model the causal relation between the content variable and the style variable and omit the spurious correlation between ĉ (X) and ŝ (X), because the spurious correlation is not a causal relation, although ĉ (X) and ŝ (X) are not statistically independent. Following previous work (LeCun et al., 1998; Bengio et al., 2013), we regard DNNs as a combination of representation learning modules and linear classifier modules. Specifically, we apply a linear function to the learned representation for approximating the content which is further used to predicting labels. Specifically, we have ĉ (X) = ĉ (r (X;Wr) ;Wc) = Wcr (X;Wr). That is, Wc are parameters used for predicting labels. According to the Gaussian distribution assumption, we can obtain the estimated style by the reparameterization trick, i.e., ŝ (X) = µ (X;Ws) + σn, where Ws presents parameters for modeling the mean, and n is sampled from a normal distribution. For simplicity, we assume that µ (X;Ws) is an affine mapping applied to the learned representation 4, i.e., µ (X;Ws) = µ (r (X;Wr) ;Ws) = Wsr (X;Wr). Assume that r (X;Wr) is a Gaussian distribution with a covariance matrix M . Then, the independence ĉ (X) ⫫ ŝ (X) holds if we set Ws as an
  4Studying non-linear functions is an interesting open question, and we leave it as future work.
  instantiation of the orthogonal complement of Wc, i.e., ker (Wc)⊥ ⊥M ker (Ws)⊥. Here, we define ⟨a, b⟩M = ⟨a,Mb⟩, and the orthogonality of two subspaces is defined likewise. Therefore, we can simply employ the learned representation r (X;Wr) and the parameters used for predicting labels, i.e., Wc, to estimate µ (X), see Appendix C for details. Combining Eq. 7 and Eq. 9, we derive a realization of adversarial distribution alignment method:
  min θ,Wg
  E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y )
  + λ (CE (g (µ (X + Eadv) ;Wg) , Y ) + βCE (g (µ (X) ;Wg) , Y )) . (11)
  Given X , Eq. 11 encourages the statistically conditional association between labels and style variables of the adversarial distribution to be close to that of the natural distribution. The explicit distribution alignment can reduce the difference between the adversarial distribution and the natural distribution. If the conditional association of the adversarial distribution is similar to that of the natural distribution, it should be hard for the adversary to find adversarial examples, which is consistent with recent work (Kilbertus et al., 2018; Schölkopf et al., 2021).
sections.heading: 4 EXPERIMENT
sections.text: In this section, we verify the efficacy of the proposed adversarial distribution alignment method by extensive experiments. Experimental results consistently demonstrate that taking into account the spurious correlation can significantly improve the adversarial robustness. In addition, the conducted adaptive attack (Tramer et al., 2020) demonstrates that the proposed method is genuinely robust.
sections.heading: 4.1 SETUPS
sections.text: Baseline methods. Our experiments are designed to demonstrate the necessity of considering the spurious correlation between labels Y and style variables S when developing robust models. Eq. 7 shows that if we set the hyperparameter γ = 0, and omit the spurious correlation between Y and S, the proposed method is equivalent to the adversarial training variant Madry (Madry et al., 2017). Thus, to demonstrate the necessity of considering the spurious correlation, we set γ = 0 and λ > 0 in Eq. 7, named ADA-M, and compare the adversarial robustness of ADA-M with that of Madry. In addition, the model capacity is often insufficient in adversarial training (Zhang et al., 2020b), so replacing one-hot labels of the first term in Eq. 7 with soft targets, i.e., the model prediction h(X;θ), can relieve the problem of insufficient model capacity 5. Considering the insufficient model capacity, we replace Y in the first term of Eq. 7 with the model prediction, and the derived method is called ADA-T. We find that ADA-T becomes the objective introduced in TRADES (Zhang et al., 2019) when we omit the spurious correlation, i.e., λ = 0, which shows that the proposed method also shares the same spirits to TRADES. Therefore, to demonstrate the importance of the spurious correlation between Y and S, we compare ADA-M and ADA-T with Madry and TRADES, respectively.
  Evaluation metrics and training details. To evaluate the robustness for different methods, we compute the test accuracy on natural and adversarial examples with `∞-norm bounded perturbation generated by: FGSM (Goodfellow et al., 2014), PGD (Madry et al., 2017), and C&W (Carlini & Wagner, 2017) attacks. The robustness is evaluated on both the best checkpoint model suggested by (Rice et al., 2020) and the last checkpoint model used in (Madry et al., 2017), respectively. For MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017; Zhang et al., 2019). For CIFAR10 and CIFAR100, two architectures are employed: ResNet-18 (He et al., 2016) and WRN34-10 (Zagoruyko & Komodakis, 2016). The settings of attacks and hyper-parameters for training are the same as previous works, more details can be found in Appendix D.
sections.heading: 4.2 ROBUSTNESS EVALUATION
sections.text: We evaluate the robustness of Madry, TRADES, and the proposed method on MNIST, CIFAR10, and CIFAR100 against various attacks (Goodfellow et al., 2014; Madry et al., 2017; Carlini & Wagner, 2017), which are widely used in the literature. We report the classification accuracy on MNIST in Table 1, where “Natural” denotes the accuracy on natural test images. We denote by PGD-40 the
  5This is because knowledge distillation (Hinton et al., 2015) demonstrates that models with insufficient capacity prefer soft targets.
  PGD attack with 40 iterations for generating adversarial examples, which also applies to the C&W attack. The results of ResNet-18 on CIFAR10 and CIFAR100 are illustrated in Table 2 and Table 3, respectively. The results of WRN-34-10 are in Appendix E. We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can significantly improve the adversarial robustness. To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Appendix F. Note that, the standard deviations of 5 runs are omitted because they hardly affect the results.
  4.3 DISCUSSION
  Consideration of gradient obfuscation. According to the criterion suggested by (Athalye et al., 2018), we exclude the potential effect of gradient obfuscation by showing the following phenomenons: a) the performance of our method on FSGM attack (59.17%) is better than iterative attacks PGD20 (51.82%) and C&W-20 (50.93%); b) the performance of our method on black-box PGD-20 (83.65%) and C&W-20 (83.57%) attacks is better than that on white-box attacks (51.82%); c) strong attacks cause lower accuracy than weak attacks, i.e., the accuracy on PGD-20 and PGD-100 are 51.82% and 48.05%, respectively. In addition, no gradient shattering operator is used in our method. All these results are evaluated on the CIFAR10 dataset using the last checkpoint of ResNet-18. Thus, according to the criterion suggested by (Athalye et al., 2018), the robustness improvement of the proposed method does not result from gradient obfuscation. Consideration of adaptive attack. According to
  the adaptive attack criterion (Tramer et al., 2020), we augment the original objective function used in the PGD attack with the proposed adversarial distribution alignment loss to implement adaptive attacks, i.e., Eq. 11. Under the adaptive attack, the accuracy is 51.68%, while under the original PGD-20 attack is 51.82%, demonstrating that the proposed method is genuinely robust.
  Consideration of AutoAttack. Following previous work (Pang et al., 2020; Wang et al., 2019), we verify the effectiveness of the proposed method on AutoAttack. According to the commonly used setting (Pang et al., 2020; Wang et al., 2019), we report robust accuracy of WRN-34-10 trained with CIFAR10 dataset on Auto-Attack, Madry: 49.58%, ADA-M: 51.56%, TRADES: 52.46%, ADA-
  T: 54.09%, HE (Pang et al., 2020): 53.74%, AWP (Wu et al., 2020): 56.17%, and MART (Wang et al., 2019): 56.29%, where HE, AWP and MART are adversarial variants achieving state-of-theart performance. These results show that the proposed method can endow models with robustness comparable to the state-of-the-art performance.
  Mitigating robust overfitting. The recent work (Rice et al., 2020) first studies the robust overfitting phenomenon. Robust overfitting means that further training will increase the robust training accuracy and test accuracy on natural data after a certain training epoch, but the robust test accuracy will decrease. The robust overfitting phenomenon of Madry (Madry et al., 2017) is depicted in Fig. 2. It can be seen that the robust test accuracy of Madry decreases to about 44%, while the best robust accuracy of Madry is 51.92%. In contrast, the proposed method drastically reduces the difference between the best robust accuracy 54.44% and the robust accuracy 50.93% of the last checkpoint.
sections.heading: 5 RELATED WORK
sections.text: A large body of work has a constructive contribution to adversarial learning and causal reasoning. We only introduce the most related papers in this section due to the limited space.
  Adversarial attack. Since the realization of the adversarial example phenomenon (Biggio et al., 2013; Szegedy et al., 2014), tons of adversarial attacks have been proposed (Moosavi-Dezfooli et al., 2016; Goodfellow et al., 2014; Carlini & Wagner, 2017; Dong et al., 2018; Tu et al., 2019; Madry et al., 2017; Croce & Hein, 2020). Among these attacks, FGSM Goodfellow et al. (2014), PGD attack (Madry et al., 2017), C&W attack (Carlini & Wagner, 2017), and Auto-Attack (Croce & Hein, 2020) are the most commonly used attacks for evaluating robustness.
  Adversarial defense. The development of adversarial attacks promotes the progress of adversarial defense and detection. Recent work on improving adversarial robustness mainly falls into two categories: certified defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) and empirical defense and detection (Najafi et al., 2019; Carmon et al., 2019; Shafahi et al., 2019; Wong et al., 2020; Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020; Ma et al., 2018; Gao et al., 2021). Detailed discussions of these exciting works can be found in Appendix G.
  Causal reasoning. The field of graphical causality, like machine learning, has a long history, see Pearl (2009); Schölkopf et al. (2021); Peters et al. (2017). One purpose of causal reasoning is to pursue the causal effect of interventions, contributing to achieving the desired objectives. Recent work shows the benefits of introducing causality into machine learning from various aspects (Zhang et al., 2020a; Mitrovic et al., 2020; Teshima et al., 2020; Tang et al., 2020; Sauer & Geiger, 2021). More details about relevant works can be found in Appendix H.
sections.heading: 6 CONCLUSION
sections.text: In this paper, we provide a novel causality viewpoint for understanding and mitigating adversarial vulnerability. Through constructing the causal graph of the adversarial data generation process and formalizing the intuition of adversarial attacks, we show that the spurious correlation between labels and style variables is important for understanding and mitigating adversarial vulnerability. Inspired by the observation, we propose the adversarial distribution alignment method, which takes the spurious correlation into account for robustness improvement. In addition, we find that the proposed method shares the same spirits to existing adversarial training variants. In future work, we will develop more effective algorithms to leverage or eliminate the spurious correlation between labels and style variables to further improve adversarial robustness. In addition, we will explore the uses of counterfactual statements to explain and mitigate the adversarial vulnerability. In sum, we make a first step towards employing causality to contribute to adversarial learning.
sections.heading: 7 ETHICS STATEMENT
sections.text: This paper does not raise any ethics concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.
sections.heading: 8 REPRODUCIBILITY STATEMENT
sections.text: To ensure the reproducibility of experimental results, we will provide a link for an anonymous repository of the source codes for this paper during the discussion phase.
sections.heading: A DERIVATION OF DISTRIBUTION DIVERGENCE
sections.text: Considering that the adversarial distribution often cannot be obtained analytically, we need to relax the distribution divergence in Eq. 4, i.e., d (P (Y ∣X) , Pθ (Y ∣X̃)). Without loss of generality, the metric d in Eq. 4 can be realized as total variation distance (TVD). Thus, according to the definition of TVD, we have
  d (P (Y ∣X) , Pθ (Y ∣X̃)) ≤ d (Pθ (Y ∣X̃) , Qθ (Y ∣X̃)) + d (P (Y ∣X) , Qθ (Y ∣X̃)) , (12) where Qθ(Y ∣X) is the conditional distribution specified by the classifier h (X;θ). According to the Pinsker inequality, i.e., d (P,Q) ≤ √ KL(P ∣∣Q)
  2 , where KL is the Kullback-Leibler divergence,
  we have an upper bound of Eq. 12:
  d (P (Y ∣X) , Pθ (Y ∣X̃)) ≤
  √ KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃))
  2 +
  √ KL (P (Y ∣X) ∣∣Qθ (Y ∣X̃))
  2 .
  (13)
  Now, we can employ the upper bound Eq. 13 as a surrogate loss. In practice, we can replace √ KL(P ∣∣Q)
  2 with KL (P ∣∣Q). The intuition is that the difference between
  √ KL(P ∣∣Q)
  2
  and KL (P ∣∣Q) is relatively small when KL (P ∣∣Q) is not extremely large, so the replacement will not introduce much difference. Thus, we derive Eq. 5: KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) + γKL (P (Y ∣X) ∣∣Qθ (Y ∣X)) , where γ is a tunable hyperparameter. Here, we give an intuitive explanation of why optimizing Eq. 5 can make the adversarial distribution similar to the natural distribution. Considering that both the adversarial distribution Pθ (Y ∣X̃) and the anchor Qθ (Y ∣X̃) are parameterized by θ, so optimizing θ to minimize KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) can force the adversarial distribution be similar to the distribution specified by the classifier h (X; θ), i.e., Qθ (Y ∣X̃). That is, optimizing Qθ (Y ∣X̃) will also change the adversarial distribution Pθ (Y ∣X̃). In addition, minimizing the divergence KL (P (Y ∣X) ∣∣Qθ (Y ∣X̃)) can endow the classifier with the ability to provide a good prediction performance. Thus, minimizing KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) will make the adversarial distribution be similar with the natural distribution.
  To verify the replacement will not introduce much difference, we evaluate the robustness of ADA-M and ADA-T on the CIFAR-10 dataset with two losses, i.e., Eq. 5 and Eq. 13. The results evaluated on best checkpoints are shown in Table 4.
sections.heading: B CALCULATION OF THE EXPECTATION ON THE STYLE INFORMATION
sections.text: We provide details of calculating Eŝ(X̃)∼N (µ(X̃),σ2I)CE (g (ŝ (X̃) ;Wg) , Y ). We assume a normal distribution for the styles, i.e., ŝ (X̃) ∼ N (µ (X̃) , σ2I). According to the definition of the crossentropy loss, for a input pair (x, y) we have:
  Eŝ(x)∼N (µ(x),σ2I)CE (g (ŝ (x) ;Wg) , y)
  = Eŝ(x)∼N (µ(x),σ2I) log 1
  P (Y = y∣g (ŝ (x) ;Wg)))
  ≤ log 1
  Eŝ(x)∼N (µ(x),σ2I)P (Y = y∣g (ŝ (x) ;Wg)))
  = log 1
  Eŝ(x)∼N (µ(x),σ2I) e W⊤g,yŝ(x)
  ∑j e W⊤g,j ŝ(x)
  = logEŝ(x)∼N (µ(x),σ2I) ∑ j
  e (Wg,j−Wg,y)⊤ŝ(x)
  = log∑ j
  e (Wg,j−Wg,y)⊤ŝ(x)+ 12 (Wg,j−Wg,y) ⊤ σ 2 I(Wg,j−Wg,y)
  = log ∑j eW
  ⊤ g,j ŝ(x)+σ
  2 2 (Wg,j−Wg,y)⊤(Wg,j−Wg,y)
  W⊤g,y ŝ(x)
  ≜ log 1 P ((Y = y∣g (ŝ(x);Wg)) ≜ CE (g (ŝ (x) ;Wg) , y) ,
  (14)
  where the inequality follows from the Jensen’s inequality: E log(X) ≤ logEX , the expectation is calculated by leveraging the moment-generating function:
  EetX = etµ+ 1 2 σ 2 t 2
  , X ∼ N (µ, σ2). (15) Note that, we define the function g (ŝ (x) ;Wg) for simplicity:
  P ((Y = y∣g (ŝ(x);Wg)) ≜ W ⊤ g,y ŝ(x)
  ∑j=1 eW ⊤ g,j ŝ(x)+σ
  2 2 (Wg,j−Wg,y)⊤(Wg,j−Wg,y)
  . (16)
  Besides seting g to a linear model, non-linear models, e.g., neural networks, can also be employed in practice. To verify the influence introduced by selecting different instantiation of model g, we compare different realizations of mode g, i.e., linear mapping and non-linear neural networks. The following results suggest that our method is relatively robust to the selection of model g. Note that, all these results are evaluated on the best checkpoint models trained on CIFAR-10 dataset.
sections.heading: C RELATIONSHIP BETWEEN ORTHOGONALITY AND STATISTICAL INDEPENDENCE
sections.text: We give the proof for the following lemma in Sec. 3.3. Note that, we use R to present the learned representation of X , and replace X with R for simplicity.
  Lemma 1. R ∈ Rd is the learned representation, where d is the number of dimension of R. Assume that R is a normal distribution with mean m and covariance matrix M . The content used for predicting labels, i.e., logits, is obtained by applying a linear functions to R, i.e., ĉ (R) =WcR, where Wc are parameters used for mapping R to logits. The style is modeled by a normal distribution, i.e., ŝ (R) = µ (R;Ws) + Σ (Y ) 1 2 n, where Ws presents parameters for modeling the mean of styles, and n is sampled from a standard normal distribution. Assume that µ (R;Ws) is a linear function, i.e., ŝ (R) = WsR + Σ (Y ) 1 2 n. Then, setting Ws as an instantiate of the orthogonal complement of Wc leads to statistical independence, i.e., ĉ (R) ⫫ ŝ (R). Here, ⫫ denotes the statistical independence, and we define ⟨a, b⟩M = ⟨a,Mb⟩ for a given semi-definite matrix M . The orthogonality A ⊥M B of two subspaces A and B is defined likewise.
  Proof. Under the assumption in Lemma 1, setting Ws as an instantiate of the orthogonal complement of Wc, we have:
  ker(Ws)⊥ ⊥M ker(Wc)⊥⟺ im(W⊤s ) ⊥M im(W⊤c )⟺ ⟨W⊤s a,W⊤c b⟩M = 0∀a, b ⟺ ⟨W⊤s a,MW⊤c b⟩ = 0∀a, b⟺WsMW⊤c = 0⟺ ERWs (R −m) (R −m)⊤W⊤c = 0 ⟺ ER,nWs (R + Σ 1 2n −m) (R −m)⊤W⊤c = 0⟺ Cov (ŝ (R) , ĉ (R)) = 0
  ⟺ ĉ (R) ⫫ ŝ (R) (17)
sections.heading: D MORE DETAILS ABOUT EVALUATION METRICS AND TRAINING DETAILS
sections.text: Evaluation metrics. For MNIST dataset, we set the maximum perturbation bound = 0.3, perturbation step size η = 0.01, and the number of iterations K = 40 for PGD and C&W attacks, which keeps the same as (Zhang et al., 2019). Following (Rice et al., 2020), we set perturbation bound = 8/255, perturbation step size η = /10, and the number of iterations K = 20 for CIFAR-10 dataset.
  Training details. For MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017; Zhang et al., 2019). Following (Zhang et al., 2019), the network is trained using SGD with 0.9 momentum for 50 epochs with an initial learning rate 0.01, and the batch size is set to 128. Hyperparameters used to craft adversarial examples for training are the same as those used for evaluation. These two networks share the same hyper-parameters: we use SGD with 0.9 momentum, weight decay 2 × 10−4, batch size 128, and an initial learning rate of 0.1. The maximum epoch is 120, and the learning rate is divided by 10 at epoch 60 and 90, respectively. To generate adversarial examples for training, we set the maximal perturbation = 8/255, the perturbation step size η = 2/255, and the number of iterationsK = 10, which is the same as (Rice et al., 2020). In all of our experiments β is set to 1.0. For ADA-M, λ is set to 1.0 and 0.5 for CIFAR-10 and CIFAR-100 datasets, respectively. For ADA-T, λ is set to 0.5 and 1.0 for CIFAR-10 and CIFAR-100 datasets, respectively.
sections.heading: E EXPERIMENTS OF WRN-34-10 ON CIFAR-10
sections.text: In Table 6, we report the accuracy of WRN-34-10 (Zagoruyko & Komodakis, 2016) of Madry, TRADES, and the proposed method on CIFAR-10 against various attacks, i.e., FGSM, PGD, and
  C&W attacks, which are widely used in the literature. Here, “Natural” denotes the accuracy of natural test images. We denote by PGD-20 the PGD attack with 20 iterations for generating adversarial examples, which also applies to the C&W attack. We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can significantly improve the adversarial robustness. Note that the standard deviations of 5 runs are omitted, because they hardly affect the results.
sections.heading: F ABLATION STUDY
sections.text: We implicitly conducted ablation studies when designing Table 1, Table 2, and Table 3. To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Table 7. Comparing Madry, TRADES, and ADA-M, we find that introducing the second (t2) and the third term (t3) can improve the robustness and that the effect of these two terms is close. Similarly, comparing TRADES and ADA-T, we see that introducing the third term (t3) can further improve the robustness.
  To further analyze the superiority of our method, we compare ADA-M with Madry to explore which kind of adversarial examples that ADA-M is more robust to. These analyses are based on the spurious perspective, as one main conclusion of our paper is that the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables. Specifically, we calculate the KL-divergence between KL (P (Y ∣h (x, s)) ∣∣P (Y ∣h (xadv, s))) for each input x, and divide samples into several (10 in our experiment) bins according to the KL-divergence. Then, we evaluate the robust accuracy of models trained with ADA-M and Madry in each bin sample, and results are depicted in Fig. 3. We can see that ADA-M is more robust to samples leading
  to large KL-divergence than Madry. According to these empirical results, we can conclude that the proposed method is more robust to samples causing a significant difference between natural and adversarial distributions.
  To further verify that ADA-T can outperform TRADES interms of both the natural and adversairal robustness, we compare the robust accuracy of ADA-T with that of TRADES on CIFAR-10 and CIFAR-100 dataset. The results evaluated on the best checkpoint models are shown in Table 8. We
  can see that ADA-T can improve both the natural and the adversarial accuracy.
  To explore the sensitivity of our method on hyperparameters, we evaluate our methods on CIFAR10 and CIFAR100 datasets. The results are organized in Table 9. Results in Table 9 demonstrate that
  both ADA-M and ADA-T are relatively insensitive to the hyperparameters.
sections.heading: G MORE DETAILS ABOUT ADVERSARIAL LEARNING
sections.text: Recent work on improving adversarial robustness mainly falls into two categories: certified defense and empirical methods.
  Certified defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) aims to endow the model with provably adversarial robustness against norm-bounded perturbations. Although the certified defense strategy is promising, the empirical defense (Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019; Wang et al., 2019; Pang et al., 2020; Wong & Kolter, 2018; Xie et al., 2019; Yang et al., 2019), especially the adversarial training method (Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019), is currently the most effective strategy. Empirical defense firstly generates adversarial examples using a certain adversarial attack, then incorporates the generated adversarial examples into the training process.
  Recently, various efforts (Najafi et al., 2019; Carmon et al., 2019; Shafahi et al., 2019; Wong et al., 2020; Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020) have been devoted to improving adversarial training. One line of work focuses on accelerating the training procedure (Shafahi et al., 2019; Wong et al., 2020). Another line of research (Najafi et al., 2019; Carmon et al., 2019) shows a promising direction that unlabeled training data can significantly mitigate the adversarial vulnerability. Lastly, recent work (Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020) provides an interesting direction where these methods rethink the adversarial training from different aspects, containing rethinking the misclassified examples (Wang et al.,
  2019), rethinking the importance weight of each example (Zhang et al., 2020b) and rethinking the role of normalization (Pang et al., 2020) and basic training strategies (Rice et al., 2020). However, all these methods overlook the spurious correlation between labels and the style information.
  Another related work is (Ilyas et al., 2019), which provides an interesting viewpoint, i.e., adversarial examples can be viewed as a human phenomenon because the model’s reliance on useful but not robust features leads to adversarial vulnerability. Our work gives a new causal perspective of adversarial vulnerability. Specifically, a) (Ilyas et al., 2019) found some features were useful but not robust, while our work explores the phenomenon’s fundamental cause and provides a clear explanation of why some features are useful but not robust: GivenX , labels Y are spuriously correlated with the style variables, so fitting the spurious correlation can predict labels. Thus, the style variables can be viewed as ‘features’; b) (Ilyas et al., 2019) claimed that adversarial examples could be viewed as a human phenomenon, while our work shows that adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon. Specifically, the adversarial vulnerability results from fitting the correlation between labels and style variables and failing to fit the causal relations, i.e., DNNs fail to extract content variables.
sections.heading: H MORE DETAILS ABOUT CAUSAL REASONING
sections.text: The most relevant work is CAMA (Zhang et al., 2020a) that aims to improve the robustness of DNNs on unseen perturbation via explicitly modeling the perturbation from a causal view. The main difference between our method and CAMA is that we focus on the adversarial vulnerability while CAMA aims to improve the robustness of unseen perturbations. In addition, CAMA assumes a hard intervention on a latent variable. It promotes robustness via modeling the perturbation in the latent space. In this paper, we employ a soft intervention and propose to penalize DNNs when the adversarial distribution is different from the natural distribution. A recent work (Bühlmann, 2020) also aims to connect robust learning and causality, but the main focus of (Bühlmann, 2020) is on out-of-distribution generalization, which is different from adversarial learning.
  Another related work is RELIC (Mitrovic et al., 2020), a regularizer used in self-supervised learning that uses the independence of mechanisms (Peters et al., 2017) and encourages DNNs to be invariant to different augmentations of the same instance. The self-supervised learning method (Mitrovic et al., 2020) also constructs a causal graph to model the data generation process, but the focus of RELIC is on the content invariant property, overlooking the importance of style information. Although (Sagawa et al., 2019) takes spurious correlations into account, (Sagawa et al., 2019) proposes using prior knowledge to group the training data to avoid the reliance on spurious correlations. Difference from (Sagawa et al., 2019), our method does not rely on prior knowledge. Another related work is CORE (Heinze-Deml & Meinshausen, 2021), a regularizer inspired by a causal graph is proposed to minimizing the variance of prediction and loss condition on label and ID information to mitigate the influence of domain shift. Different from (Heinze-Deml & Meinshausen, 2021), our method is designed to eliminate the difference between the natural and adversarial distributions.