summaries: The work presents a causal perspective of adversarial attacks on image-based machine learning models by studying a causal graph of the adversarial data creation process and highlighting how such a process makes the learned models vulnerable. It argues that the main reason for adversarial vulnerability is the reliance of models on spurious correlations between labels and style. Accordingly, it proposes a method to learn models for which the conditional distribution of label given style and image does not vary much when attacked. Empirically, the method is shown to be more robust than two baselines on three datasets.

summaries: This paper presents a causal perspective on addressing adversarially vulnerability. It first constructs a causal graph, which then inspires the design of the distribution alignment method for reducing the gap between adversarial and natural data. Extensive experiments on CIFAR10, CIFAR100, and MNIST demonstrate the robustness of the proposed method against various attack methods.

summaries: The paper shows a causal perspective to the adversarial robustness problem. It creates a graph over content and style variable sets. It identifies the spurious correlation between style and label as the main reason for adversarial examples, and then proposes a method to remove it from the trained model. Experiments on three datasets show that the proposed method is better than two baselines.

summaries: This paper proposes a causal graph to model the generation process of adversarial attacks. Based on the proposed causal graph, the authors identify the origin of adversarial vulnerability as the spurious correlation between style variable and class label. Under the adversarial distribution, such spurious correlation can be maliciously used to mislead a victim model. In this light, the authors propose a method to align the adversarial distribution and the natural distribution to prevent a model from learning spurious correlation. The proposed method is empirically validated on prevailing datasets under several attacks.

abstractText: The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. As causal reasoning has an instinct for modeling distribution change, it is essential to incorporate causality into analyzing this specific type of distribution change induced by adversarial attacks. However, causal formulations of the intuition of adversarial attacks and the development of robust DNNs are still lacking in the literature. To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and define the adversarial distribution to formalize the intuition of adversarial attacks. From the causal perspective, we study the distinction between the natural and adversarial distribution and conclude that the origin of adversarial vulnerability is the focus of models on spurious correlations. Inspired by the causal understanding, we propose the adversarial distribution alignment method to eliminate the difference between natural and adversarial distributions by considering spurious correlations. Extensive experiments demonstrate the efficacy of the proposed method. Our work is the first attempt towards using causality to understand and mitigate adversarial vulnerability.

authors:
Anish Athalye
Nicholas Carlini
David Wagner

title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

venue: In International Conference on Machine Learning,

authors:
Yoshua Bengio
Aaron Courville
Pascal Vincent

title: Representation learning: A review and new perspectives

venue: IEEE transactions on pattern analysis and machine intelligence,

authors:
Battista Biggio
Igino Corona
Davide Maiorca
Blaine Nelson
Nedim Šrndić
Pavel Laskov
Giorgio Giacinto
Fabio Roli

title: Evasion attacks against machine learning at test time

venue: In Joint European conference on machine learning and knowledge discovery in databases,

title: Invariance, causality and robustness

venue: Statistical Science,

authors:
Nicholas Carlini
David Wagner

title: Towards evaluating the robustness of neural networks

venue: In 2017 ieee symposium on security and privacy (sp),

authors:
Yair Carmon
Aditi Raghunathan
Ludwig Schmidt
Percy Liang
John C Duchi

title: Unlabeled data improves adversarial robustness

venue: arXiv preprint arXiv:1905.13736,

authors:
Ting Chen
Simon Kornblith
Mohammad Norouzi
Geoffrey Hinton

title: A simple framework for contrastive learning of visual representations

venue: In International conference on machine learning,

authors:
Juan Correa
Elias Bareinboim

title: A calculus for stochastic interventions: Causal effect identification and surrogate experiments

venue: In Proceedings of the AAAI Conference on Artificial Intelligence,

authors:
Francesco Croce
Matthias Hein

title: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

authors:
Yinpeng Dong
Fangzhou Liao
Tianyu Pang
Hang Su
Jun Zhu
Xiaolin Hu
Jianguo Li

title: Boosting adversarial attacks with momentum

venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,

authors:
Frederick Eberhardt
Richard Scheines

title: Interventions and causal inference

venue: Philosophy of science,

authors:
Yarin Gal
Zoubin Ghahramani

title: Dropout as a bayesian approximation: Representing model uncertainty in deep learning

venue: In international conference on machine learning,

authors:
Yaroslav Ganin
Evgeniya Ustinova
Hana Ajakan
Pascal Germain
Hugo Larochelle
François Laviolette
Mario Marchand
Victor Lempitsky

title: Domain-adversarial training of neural networks. The journal of machine learning

authors:
Ruize Gao
Feng Liu
Jingfeng Zhang
Bo Han
Tongliang Liu
Gang Niu
Masashi Sugiyama

title: Maximum mean discrepancy test is aware of adversarial attacks

authors:
Robert Geirhos
Patricia Rubisch
Claudio Michaelis
Matthias Bethge
Felix A Wichmann
Wieland Brendel

title: Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness

venue: In International Conference on Learning Representations,

authors:
Ian J Goodfellow
Jonathon Shlens
Christian Szegedy

title: Explaining and harnessing adversarial examples

venue: arXiv preprint arXiv:1412.6572,

authors:
Alison Gopnik
Clark Glymour
David M Sobel
Laura E Schulz
Tamar Kushnir
David Danks

title: A theory of causal learning in children: causal maps and bayes nets

venue: Psychological review,

authors:
Kaiming He
Xiangyu Zhang
Shaoqing Ren
Jian Sun

title: Deep residual learning for image recognition

authors:
Kaiming He
Haoqi Fan
Yuxin Wu
Saining Xie
Ross Girshick

title: Momentum contrast for unsupervised visual representation learning

venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,

authors:
Christina Heinze-Deml
Nicolai Meinshausen

title: Conditional variance penalties and domain shift robustness

venue: Machine Learning,

authors:
Jan Hendrik Metzen
Mummadi Chaithanya Kumar
Thomas Brox
Volker Fischer

title: Universal adversarial perturbations against semantic image segmentation

authors:
Geoffrey Hinton
Oriol Vinyals
Jeff Dean

title: Distilling the knowledge in a neural network

venue: arXiv preprint arXiv:1503.02531,

authors:
Andrew Ilyas
Shibani Santurkar
Logan Engstrom
Brandon Tran
Aleksander Madry

title: Adversarial examples are not bugs, they are features

authors:
Alex Kendall
Yarin Gal

title: What uncertainties do we need in bayesian deep learning for computer vision

venue: In Proceedings of the 31st International Conference on Neural Information Processing Systems,

authors:
Niki Kilbertus
Giambattista Parascandolo
Bernhard Schölkopf

title: Generalization in anti-causal learning

venue: arXiv preprint arXiv:1812.00524,

authors:
Alex Krizhevsky
Geoffrey Hinton

title: Learning multiple layers of features from tiny images

authors:
Alex Krizhevsky
Ilya Sutskever
Geoffrey E Hinton

title: Imagenet classification with deep convolutional neural networks

authors:
Yann LeCun
Léon Bottou
Yoshua Bengio
Patrick Haffner

title: Gradient-based learning applied to document recognition

venue: Proceedings of the IEEE,

authors:
Xingjun Ma
Bo Li
Yisen Wang
Sarah M Erfani
Sudanthi Wijewickrema
Grant Schoenebeck
Dawn Song
Michael E Houle
James Bailey

title: Characterizing adversarial subspaces using local intrinsic dimensionality

venue: In ICLR,

authors:
Aleksander Madry
Aleksandar Makelov
Ludwig Schmidt
Dimitris Tsipras
Adrian Vladu

title: Towards deep learning models resistant to adversarial attacks

venue: arXiv preprint arXiv:1706.06083,

authors:
Jovana Mitrovic
Brian McWilliams
Jacob C Walker
Lars Holger Buesing
Charles Blundell

title: Representation learning via invariant causal mechanisms

authors:
Seyed-Mohsen Moosavi-Dezfooli
Alhussein Fawzi
Pascal Frossard

title: Deepfool: a simple and accurate method to fool deep neural networks

authors:
Seyed-Mohsen Moosavi-Dezfooli
Alhussein Fawzi
Omar Fawzi
Pascal Frossard

title: Universal adversarial perturbations

authors:
Amir Najafi
Shin-ichi Maeda
Masanori Koyama
Takeru Miyato

title: Robustness to adversarial perturbations in learning from incomplete data

authors:
Tianyu Pang
Xiao Yang
Yinpeng Dong
Kun Xu
Jun Zhu
Hang Su

title: Boosting adversarial training with hypersphere embedding

venue: arXiv preprint arXiv:2002.08619,

authors:
Nicolas Papernot
Patrick McDaniel
Ian Goodfellow
Somesh Jha
Z Berkay Celik
Ananthram Swami

title: Practical black-box attacks against machine learning

venue: In Proceedings of the 2017 ACM on Asia conference on computer and communications security,

authors:
Jonas Peters
Dominik Janzing
Bernhard Schölkopf

title: Elements of causal inference: foundations and learning algorithms

authors:
Aditi Raghunathan
Jacob Steinhardt
Percy Liang

title: Certified defenses against adversarial examples

venue: arXiv preprint arXiv:1801.09344,

authors:
Nasim Rahaman
Aristide Baratin
Devansh Arpit
Felix Draxler
Min Lin
Fred Hamprecht
Yoshua Bengio
Aaron Courville

title: On the spectral bias of neural networks

authors:
Leslie Rice
Eric Wong
Zico Kolter

title: Overfitting in adversarially robust deep learning

authors:
Shiori Sagawa
Pang Wei Koh
Tatsunori B Hashimoto
Percy Liang

title: Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization

authors:
Axel Sauer
Andreas Geiger

title: Counterfactual generative networks

venue: arXiv preprint arXiv:2101.06046,

authors:
Bernhard Schölkopf
Francesco Locatello
Stefan Bauer
Nan Rosemary Ke
Nal Kalchbrenner
Anirudh Goyal
Yoshua Bengio

title: Toward causal representation learning

authors:
Ali Shafahi
Mahyar Najibi
Amin Ghiasi
Zheng Xu
John Dickerson
Christoph Studer
Larry S Davis
Gavin Taylor
Tom Goldstein

title: Adversarial training for free

authors:
Karen Simonyan
Andrew Zisserman

title: Very deep convolutional networks for large-scale image recognition

venue: arXiv preprint arXiv:1409.1556,

authors:
Sahil Singla
Soheil Feizi

title: Second-order provable defenses against adversarial attacks

authors:
Christian Szegedy
Wojciech Zaremba
Ilya Sutskever
Joan Bruna
Dumitru Erhan
Ian J. Goodfellow
Rob Fergus

title: Intriguing properties of neural networks

venue: In 2nd International Conference on Learning Representations,

authors:
Christian Szegedy
Wei Liu
Yangqing Jia
Pierre Sermanet
Scott Reed
Dragomir Anguelov
Dumitru Erhan
Vincent Vanhoucke
Andrew Rabinovich

title: Going deeper with convolutions

authors:
Kaihua Tang
Jianqiang Huang
Hanwang Zhang

title: Long-tailed classification by keeping the good and removing the bad momentum causal effect

venue: Advances in Neural Information Processing Systems,

authors:
Takeshi Teshima
Issei Sato
Masashi Sugiyama

title: Few-shot domain adaptation by causal mechanism transfer

authors:
Florian Tramer
Nicholas Carlini
Wieland Brendel
Aleksander Madry

title: On adaptive attacks to adversarial example defenses

authors:
Chun-Chen Tu
Paishun Ting
Pin-Yu Chen
Sijia Liu
Huan Zhang
Jinfeng Yi
Cho-Jui Hsieh
Shin-Ming Cheng

title: Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks

authors:
Cumhur Erkan Tuncali
Georgios Fainekos
Hisahiro Ito
James Kapinski

title: Simulation-based adversarial test generation for autonomous vehicles with machine learning components

venue: IEEE Intelligent Vehicles Symposium (IV),

authors:
Yisen Wang
Difan Zou
Jinfeng Yi
James Bailey
Xingjun Ma
Quanquan Gu

title: Improving adversarial robustness requires revisiting misclassified examples

authors:
Eric Wong
Zico Kolter

title: Provable defenses against adversarial examples via the convex outer adversarial polytope

authors:
Eric Wong
Leslie Rice
J Zico Kolter

title: Fast is better than free: Revisiting adversarial training

venue: arXiv preprint arXiv:2001.03994,

authors:
Dongxian Wu
Shu-Tao Xia
Yisen Wang

title: Adversarial weight perturbation helps robust generalization

venue: arXiv preprint arXiv:2004.05884,

authors:
Cihang Xie
Yuxin Wu
Laurens van der Maaten
Alan L Yuille
Kaiming He

title: Feature denoising for improving adversarial robustness

authors:
Yuzhe Yang
Guo Zhang
Dina Katabi
Zhi Xu

title: Me-net: Towards effective adversarial robustness with matrix estimation

title: Wide residual networks. In British Machine Vision Conference 2016

venue: British Machine Vision Association,

authors:
Cheng Zhang
Kun Zhang
Yingzhen Li

title: A causal view on robustness of neural networks

authors:
Chiyuan Zhang
Samy Bengio
Moritz Hardt
Benjamin Recht
Oriol Vinyals

title: Understanding deep learning (still) requires rethinking generalization

venue: Communications of the ACM,

authors:
Hongyang Zhang
Yaodong Yu
Jiantao Jiao
Eric Xing
Laurent El Ghaoui
Michael Jordan

title: Theoretically principled trade-off between robustness and accuracy

authors:
Jingfeng Zhang
Jianing Zhu
Gang Niu
Bo Han
Masashi Sugiyama
Mohan Kankanhalli

title: Geometry-aware instance-reweighted adversarial training

venue: arXiv preprint arXiv:2010.01736,

authors:
Kun Zhang
Bernhard Schölkopf
Krikamol Muandet
Zhikun Wang

title: Domain adaptation under target and conditional shift

authors:
Yonggang Zhang
Xinmei Tian
Ya Li
Xinchao Wang
Dacheng Tao

title: Principal component adversarial example

title: Following (Zhang et al., 2019), the network is trained using SGD

authors:
Xie et al
Yang

title: 2019), especially the adversarial training method (Goodfellow

heading: 1 INTRODUCTION

text:
The seminal work (Szegedy et al., 2014; Biggio et al., 2013) shows that DNNs are vulnerable to adversarial examples, which consist of malicious perturbations imperceptible to humans yet fooling state-of-the-art models (Krizhevsky et al., 2012; Szegedy et al., 2015; Simonyan & Zisserman, 2014; He et al., 2016). The lack of robustness hinders the applications of DNNs to some safetycritical areas such as automatic driving (Tuncali et al., 2018) and healthcare (Finlayson et al., 2019). Therefore, mitigating the adversarial vulnerability is critical to the further development of DNNs.
Human cognitive systems are immune to the distribution change induced by adversarial attacks because humans are more sensitive to causal relations than statistical associations (Gopnik et al., 2004). Using causal language, causal reasoning can identify causal relation and ignore nuisance factors, i.e., not the cause of labels, by intervention (Pearl, 2009; Peters et al., 2017). As adversarial perturbations are usually imperceptible and make no impact on human decisions (Szegedy et al., 2014; Goodfellow et al., 2014), it is reasonable to assume that the difference between natural and adversarial examples comes from nuisance factors. This is because if task-relevant factors of some samples are changed, these samples will make both humans and DNNs change their decisions. From a causal viewpoint (Zhang et al., 2020a), adversarial attacks can be regarded as a specific type of distribution change resulting from the intervention on the natural data distribution. In summary, causal reasoning has the instinct for analyzing the effect of the intervention caused by adversarial attacks, so it is essential to leverage causality to understand and mitigate the adversarial vulnerability.
However, there are two significant problems to overcome before using causality to understand and mitigate the adversarial vulnerability. Firstly, constructing a causal graph is arguably the fundamental premise for causal reasoning (Pearl, 2009; Peters et al., 2017), but how to construct causal graphs in the context of adversarial attacks is still lacking in the literature. Secondly, using causal language to formalize the intuition of adversarial attacks is the key to connect causality and adversarial vulnerability, but it also remains to be solved. These two problems are fundamental obstacles, which prevent us from employing causality to contribute to adversarial learning.
To address these challenges, we first construct a causal graph to model the perceived data generation process where nuisance factors are considered. The constructed causal graph can entail a specific intervention distribution, i.e., the adversarial distribution. Moreover, the causal graph immediately shows that, given inputs, labels are statistically correlated with nuisance factors, which have no cause-effect to labels. The spurious correlation implies that if DNNs fit the conditional association between labels and nuisance factors, their performance on different conditional associations between labels and nuisance factors will change accordingly. Through investigating the distinction between these two distributions induced by nuisance factors, we conclude that adversarial distributions can be obtained by exploiting conditional associations between labels and nuisance factors, where the conditional association of adversarial distributions is drastically different from that of natural distributions. Namely, the origin of adversarial vulnerability is the focus of DNNs on the spurious correlation between labels and nuisance factors.
According to the causal perspective, an adversarial distribution is crafted by exploiting a specific conditional association between labels and nuisance factors, this association is drastically different from that of natural distributions. Intuitively, eliminating the difference in such conditional associations between natural and adversarial distributions can promote the performance of models on adversarial distributions. Thus, we propose an adversarial distribution alignment method to eliminate the difference between these two distributions. Surprisingly, we find that the proposed method shares the same spirits to existing adversarial training (Goodfellow et al., 2014) variants, i.e., Madry (Madry et al., 2017) and TRADES (Zhang et al., 2019). We validate the efficacy of the proposed method on MNIST, CIFAR10, and CIFAR100 (Krizhevsky et al., 2009) datasets under various adversarial attacks such as FGSM (Goodfellow et al., 2014), PGD (Madry et al., 2017), CW attack (Carlini & Wagner, 2017), and AutoAttack (Croce & Hein, 2020). Extensive experiments demonstrate that the proposed method can improve the adversarial robustness significantly.
Our main contributions are:
• We provide a causal perspective to understand and mitigate the adversarial vulnerability, which is the first attempt towards using causality to contribute adversarial learning.
• To leverage causality to contribute adversarial learning, we solve two fundamental problems. Specifically, we construct a causal graph to model the adversarial data generation process and define the adversarial distribution to formalize adversarial attacks.
• A defense method called adversarial distribution alignment is proposed to reduce adversarial vulnerability by eliminating the difference between adversarial distribution and natural distribution. Extensive experiments demonstrate that the proposed method can significantly improve the adversarial robustness.

heading: 2 A CAUSAL VIEW ON ADVERSARIAL DATA GENERATION

text:
The ability of humans to perform causal reasoning is arguably an essential factor that makes human learning different from deep learning (Schölkopf et al., 2021; Zhang et al., 2020a; Gopnik et al., 2004). The superiority of causal reasoning endows humans with the ability to identify causal relations and ignore nuisance factors that are not relevant to the task. In contrast, DNNs are usually trained to fit the perceived information overlooking the ability to distinguish causal relations and statistical associations. This shortcut solution could lead to overfitting to these nuisance factors, which would further result in the sensitivity of DNNs to such factors. Therefore, we propose incorporating causal reasoning to mitigate the sensitivity of DNNs to these nuisance factors.
Before using causal reasoning to analyze adversarial vulnerability, we need to construct a causal graph, as causal graphs are the key for formalizing causal reasoning (Peters et al., 2017). In the context of adversarial learning, we desire the causal graph by which both the natural and the adversarial distributions can be generated. In addition, the graph is required to reflect the impact of nuisance factors on these two distributions, so that we can investigate the difference in nuisance factors between these two distributions. Consequently, we can formally establish the connection between nuisance factors and adversarial vulnerability. Therefore, we propose constructing a causal graph to model the adversarial generation process where the nuisance factors are considered. One approach is to use causal structure learning to infer causal graphs (Pearl, 2009; Peters et al., 2017), but it is challenging to apply this kind of approach to high-dimensional data. Using external knowl-
edge to construct causal graphs is another approach (Zhang et al., 2013; Tang et al., 2020; Schölkopf et al., 2021). As automatically learning a precise causal graph is out of scope for this work, external human knowledge about the data generation process is employed to construct the causal graph.
Specifically, we construct a causal graph G to formalize the image data generation process using the following knowledge for analyzing adversarial vulnerability. As there might be a number of different causes of natural data X , we propose to divide all the causes into two categories for simplicity. We group content-related causes into one category, called content variable C. The rest causes, i.e., nuisance factors, are grouped into another category, called style variable S, which is contentindependent, i.e., S ⫫ C. This implies that C → X ← S. It is noteworthy that, in this paper, we assume that only the content variable is relevant for the task we care about, i.e., C → Y . Perceived data X̃ are usually composed of perturbations E and natural data X . When the perturbation E is designed carefully to fool DNNs, E should be a compound result of X (the object to be perturbed), Y (the reference for the perturbation), and θ (the targets affected by the perturbation), e.g., white-box attacks (Goodfellow et al., 2014; Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016), which means that (X,Y,θ) → E 1. Leverage all this background knowledge, we obtain the causal graph G formalizing the perturbed data generation process, depicted in Fig. 1. Based on the causal graph, we can define valid interventions (Pearl, 2009; Schölkopf et al., 2021) and the corresponding intervention distributions. Defining valid interventions is equivalent to determining which variables or mechanisms in the causal graph can be intervened. In this work, we consider both hard and soft interventions (Eberhardt & Scheines, 2007; Correa & Bareinboim, 2020) on the perturbation variable E. Specifically, we can use a structural causal model to represent the generating mechanism of E: E ≔M(X,Y,θ, UE), (1) where the exogenous variable UE stands for other indeterminacies, e.g., random start noise used in PGD attack (Madry et al., 2017).
By intervening E in different ways, we can obtain different intervention distributions, which could be either natural or adversarial, over the observed variables. For example, if we do a hard intervention on E, i.e., do(E = 0) in the graph, the generated data distribution corresponds to the natural distribution P (X,Y ). Different perturbations can be obtained by performing soft interventions on E, i.e., modifying M. In the context of adversarial attacks, adversaries aim to maximize a certain objective function `(⋅) to mislead a target model h (X;θ) by searching for the worst perturbation for each instance (Goodfellow et al., 2014; Carlini & Wagner, 2017; Dong et al., 2018; Madry et al., 2017), where θ stands for parameters of the target model. To formalize the intuition of adversarial attacks, we can first search for the adversarial perturbation Eadv , and then use a function to approximate the mechanism for generating Eadv . The adversarial perturbation Eadv can be obtained by maximizing `(⋅):
Eadv = arg max E ′∈B
`(h(X + E ′;θ), Y ). (2)
1These three causes are not indispensable. Deleting θ gives black-box attacks (Papernot et al., 2017; Dong et al., 2018) that only use (X,Y ) to perform attacks. Deleting X leads to universal adversarial attacks (Moosavi-Dezfooli et al., 2017; Hendrik Metzen et al., 2017), which assume that one adversarial perturbation is sufficient to fool DNNs. Recent works show that a specific transform to X effectively evaluates the sensitivity of DNNs within a small neighborhood (Rahaman et al., 2019; Zhang et al., 2020c), which corresponds to deleting (Y,θ) from the cause set.
where B is a set of valid perturbations, and the adversarial perturbation Eadv is the result of the mechanism Madv , i.e.,Eadv ≔Madv(X,Y,θ, UE). The intervention distribution that corresponds to the adversarial mechanism Madv is defined as the adversarial distribution Pθ(X̃, Y ), where the subscript θ indicates that Pθ(X̃, Y ) is crafted to attack the target model h (X;θ) with parameter θ.

heading: 3 METHOD

text: To understand the adversarial vulnerability, we study the difference between the natural and adversarial distributions and conclude that conditional associations between labels and style factors play a crucial role in adversarial vulnerability. Inspired by the conclusion, we propose a method to eliminate the difference in such conditional associations to mitigate adversarial vulnerability.

heading: 3.1 ORIGIN OF ADVERSARIAL VULNERABILITY

text:
Inspired by the causal graph, we derive a causal understanding of adversarial vulnerability. According to the causal graph G depicted in Fig. 1, there is a path, S → X ← C → Y , from the style variable S to the label Y when X is given 2, which leads to the correlation between labels and style variables. The spurious correlation implies that DNNs can perform well on the training set by fitting the statistical association between Y and S, even though the genuine content information is dropped. Moreover, if the conditional association between Y and S on the (natural) test set is similar to that on the training set, fitting these spurious correlations will also perform well on the test set. This is consistent with the recent work (Ilyas et al., 2019), which shows that training DNNs with incorrectly labeled data, i.e., the genuine content information is not utilized for training, yields good accuracy on the (natural) test set. However, suppose DNNs learn such spurious correlation. In that case, their performance will change with the conditional association between Y and S. Consequently, fitting such spurious correlation gives adversaries a chance to fool DNNs. Therefore, the conditional association between labels and style variables is a key to understand adversarial vulnerability.
To identify the origin of adversarial vulnerability from the spurious correlation perspective, we study the distinction between the adversarial distribution and the natural distribution. To look at what makes the adversarial distribution different from the natural distribution, we expand the natural P (Y ∣X) and the adversarial distribution Pθ(Y ∣X̃)
P (Y ∣X) = ∑ s∈S P (s∣X)P (Y ∣X, s), Pθ(Y ∣X̃) = ∑ s∈S Pθ(s∣X̃)Pθ(Y ∣X̃, s), (3)
where we assume the set of valid styles S is discrete, and s stands for a certain style. We can see that the difference between P (Y ∣X) and Pθ(Y ∣X̃) results from two terms, i.e., Pθ(s∣X̃) and Pθ(Y ∣X̃, s). Pθ(s∣X̃) represents the change of style information, e.g., textures (Geirhos et al., 2018), transformations (Chen et al., 2020; Mitrovic et al., 2020; He et al., 2020), and domain shifts (Ganin et al., 2016). It is shown that changing image textures has a significant impact on predictions of DNNs (Geirhos et al., 2018), but such drastic style changes will hardly appear in the context of adversarial attacks as perturbations are required to be imperceptible 3. Although modifying style variables S is not allowed, adversaries can exploit the conditional association between Y and S, i.e., Pθ(Y ∣X̃, s), via injecting a specific perturbation to generate adversarial distributions. Specifically, under an appropriate soft intervention, i.e., Madv , the conditional distribution Pθ(Y ∣X̃, s) can be drastically different from P (Y ∣X, s). Namely, the property of adversarial distribution essentially results from the drastic difference in the statistical association between labels and style variables. Intuitively, if DNNs fit the conditional associations between Y and S, their performance will change with the spurious correlation. Consequently, a drastic difference between the conditional association between Y and S will cause significant performance degradation of DNNs.
Hence, the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables. This conclusion provides a causal perspective for the empirical
2Here, we use the underlined variable X to represent that the variable X is given. 3In this work, we focus on imperceptible adversarial examples, i.e., the widely studied `p-norm bounded
adversarial examples, where drastic style changes hardly appear.
observation, i.e. some features are useful but not robust (Ilyas et al., 2019), so adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon.

heading: 3.2 THE ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD

text:
In light of the causal understanding of the adversarial vulnerability, improving adversarial robustness requires forcing DNNs to fit the causal relations rather than merely the statistical associations. However, only the perceived data can be observed in practice, and the supervised information of content variables is usually unavailable. Therefore, an approach that can avoid reliance on such supervised information is required.
To develop such an approach, we revisit the difference between the adversarial and natural distributions. Intuitively, if the difference between the natural and the adversarial distribution is negligible, the adversarial vulnerability of DNNs can be mitigated, as DNNs perform well on the natural distribution (Krizhevsky et al., 2012; He et al., 2016; Zhang et al., 2021). Consequently, a straightforward solution for improving adversarial robustness is to align these two distributions. The aforementioned analysis shows that the property of adversarial distributions to fool DNNs comes from specific conditional associations between labels Y and style variables S, i.e., Pθ(Y ∣X̃, s) and P (Y ∣X, s). Inspired by the conclusion, we propose an adversarial distribution alignment method to eliminate the difference between the natural and adversarial distributions. Specifically, we regard the natural distribution P (Y ∣X, s) as an anchor, then align the adversarial distribution Pθ(Y ∣X̃, s) with the anchor such that the difference between these two distributions is negligible. In addition, the relationship between Y and X should also be the same on both the adversarial and natural distributions. Concretely, we operationalize the intuition of the adversarial distribution alignment method by:
min θ
d (P (Y ∣X) , Pθ (Y ∣X̃)) + λEsd (P (Y ∣X, s) , Pθ (Y ∣X̃, s)) . (4)
Here d(⋅) is a metric reflecting the divergence between two distributions, and λ > 0 presents the weighting of the misalignment penalty. In addition, we assume X and X̃ have the same support set because adversarial perturbations are usually bounded. Solving the adversarial distribution alignment objective is equivalent to search a classifier h(X̃;θ) such that the adversarial distribution of the classifier is similar to the natural distribution. Benefiting from the consistency of these two distributions, classifier h(X̃;θ) can perform well on both the natural and adversarial distributions. In practice, the adversarial distribution often cannot be obtained analytically, so we relax the distribution divergence in Eq. 4 to the sum of two divergences , see Appendix for details A. For the divergence between P (Y ∣X) and Pθ(Y ∣X̃), we have:
KL (Pθ (Y ∣X̃) , Qθ (Y ∣X̃)) + γKL (P (Y ∣X) , Qθ (Y ∣X)) =E(X̃,Y )∼Pθ(X̃,Y )CE (h (X̃;θ) , Y ) + γE(X,Y )∼P (X,Y )CE (h (X;θ) , Y ) ≈E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y ) , (5)
where KL is the Kullback-Leibler divergence, CE is the cross-entropy loss, Qθ(Y ∣X) is the conditional distribution specified by the classifier h (X;θ), and γ is a tunable hyperparameter. Because adversarial examples are usually generated by adding perturbation to their corresponding natural samples rather than sampled from the adversarial distribution independently, we use X + Eadv to approximate examples independently sampled from the adversarial distribution, whereX is sampled from the natural distribution independently.
Similar to Eq. 5, the divergence between P (Y ∣X̃, s) and Pθ(Y ∣X̃, s) can be approximated by: E(X,Y )∼P (X,Y )CE (g (s (X + Eadv) ;Wg) , Y ) + β (g (s (X) ;Wg) , Y ) , (6)
where g (s (X) ;Wg) is a function used for modeling the statistically conditional association between labels and style variables, s (X) stands for the integrated representation of X and s, and β is a tunable hyperparameter. Thus, the overall objective of the proposed adversarial distribution alignment method can be expressed as
min θ,Wg
E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y )
+ λ (EsCE (g (s (X + Eadv) ;Wg) , Y ) + βCE (g (s (X) ;Wg) , Y )) . (7)
To make sure that introducing g can benefit learning h, it is necessary to design an approach to connect model g and model h. In this paper, we connect g and h by representation sharing. Interestingly, according to Eq. 4 and Eq. 7, if we omit the spurious correlation between labels and style variables, i.e., λ = 0, and set γ = 0, Eq. 7 then becomes the objective function introduced by Madry (Madry et al., 2017). This suggests that the proposed adversarial distribution alignment method is consistent with the seminal variant Madry (Madry et al., 2017) of adversarial training.

heading: 3.3 REALIZATION OF ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD

text:
According to Eq. 7, realizing the proposed adversarial distribution alignment method requires predicting labels with all style variables, i.e., modeling the statistically conditional association between Y and S. However, there are two major obstacles preventing us from calculating the last term of Eq. 7. Specifically, a) the number of all possible styles s is infinite, so the cost of calculating the expectation in Eq. 7 can grow to infinity; b) the representation of the integrated representation of X and s, i.e., s (X), used for predicting labels is unknown. To calculate the expectation in Eq. 7, we have to make some assumptions to approximate the distribution of the style variable, as the true distribution is unknown. Following previous work (Gal & Ghahramani, 2016; Kendall & Gal, 2017), we assume a Gaussian distribution to approximate the unknown distribution. Specifically, the representation s (X) is estimated by ŝ (X) sampled from a Gaussian distribution, i.e., ŝ (X) ∼ N (µ (X) ,Σ), where µ (X) is employed to model the Gaussian distribution’s mean, and Σ is the covariance matrix. Because little prior knowledge about the covariance matrix is observed, we simply regard the covariance matrix Σ as an identity matrix, i.e., ŝ (X) ∼ N (µ (X) , σ2I). Then, the expectation can be approximated by
EsCE (g (s (X) ;Wg) , Y ) ≈ Eŝ(X)∼N (µ(X),σ2I)CE (g (ŝ (X) ;Wg) , Y ) . (8) Thanks to the Gaussian distribution assumption, we can derive an upper bound of the expectation:
Eŝ(X)∼N (µ(X),σ2I)CE (g (ŝ (X) ;Wg) , Y ) ≤ CE (g (µ (X) ;Wg) , Y ) , (9)
where the probability of the ith category of g (µ (X) ;Wg) is (see Appendix B for details)
P (Y = i∣g (µ (X) ;Wg)) = e W ⊤ g,iµ(X)
∑j eW ⊤ g,jµ(X)+σ
2 2 (Wg,j−Wg,i)⊤(Wg,j−Wg,i)
. (10)
Here, we simply set g to a linear function, i.e., Wg is a linear mapping. Instead of calculating all styles ŝ (X), the conclusion of Eq. 9 and Eq. 10 shows that only the mean and variance of ŝ (X) are required to calculate the expectation in Eq. 7.
In light of Eq. 9 and Eq. 10, the challenge of learning the representation of s (X) boils down to estimating the mean of ŝ (X), i.e., no need to estimate every style representation explicitly. In addition, the variance can be treated as a hyperparameter. To estimate the mean, we draw inspiration from the proposed causal graph. According to the causal graph G, the content and style variables are statistically independent, i.e., C ⫫ S. Thus, the estimated content ĉ (X) and style ŝ (X) are desired to be independent, i.e., ĉ (X) ⫫ ŝ (X). The underlying intuition is that we aim to model the causal relation between the content variable and the style variable and omit the spurious correlation between ĉ (X) and ŝ (X), because the spurious correlation is not a causal relation, although ĉ (X) and ŝ (X) are not statistically independent. Following previous work (LeCun et al., 1998; Bengio et al., 2013), we regard DNNs as a combination of representation learning modules and linear classifier modules. Specifically, we apply a linear function to the learned representation for approximating the content which is further used to predicting labels. Specifically, we have ĉ (X) = ĉ (r (X;Wr) ;Wc) = Wcr (X;Wr). That is, Wc are parameters used for predicting labels. According to the Gaussian distribution assumption, we can obtain the estimated style by the reparameterization trick, i.e., ŝ (X) = µ (X;Ws) + σn, where Ws presents parameters for modeling the mean, and n is sampled from a normal distribution. For simplicity, we assume that µ (X;Ws) is an affine mapping applied to the learned representation 4, i.e., µ (X;Ws) = µ (r (X;Wr) ;Ws) = Wsr (X;Wr). Assume that r (X;Wr) is a Gaussian distribution with a covariance matrix M . Then, the independence ĉ (X) ⫫ ŝ (X) holds if we set Ws as an
4Studying non-linear functions is an interesting open question, and we leave it as future work.
instantiation of the orthogonal complement of Wc, i.e., ker (Wc)⊥ ⊥M ker (Ws)⊥. Here, we define ⟨a, b⟩M = ⟨a,Mb⟩, and the orthogonality of two subspaces is defined likewise. Therefore, we can simply employ the learned representation r (X;Wr) and the parameters used for predicting labels, i.e., Wc, to estimate µ (X), see Appendix C for details. Combining Eq. 7 and Eq. 9, we derive a realization of adversarial distribution alignment method:
min θ,Wg
E(X,Y )∼P (X,Y )CE (h (X + Eadv;θ) , Y ) + γCE (h (X;θ) , Y )
+ λ (CE (g (µ (X + Eadv) ;Wg) , Y ) + βCE (g (µ (X) ;Wg) , Y )) . (11)
Given X , Eq. 11 encourages the statistically conditional association between labels and style variables of the adversarial distribution to be close to that of the natural distribution. The explicit distribution alignment can reduce the difference between the adversarial distribution and the natural distribution. If the conditional association of the adversarial distribution is similar to that of the natural distribution, it should be hard for the adversary to find adversarial examples, which is consistent with recent work (Kilbertus et al., 2018; Schölkopf et al., 2021).

heading: 4 EXPERIMENT

text: In this section, we verify the efficacy of the proposed adversarial distribution alignment method by extensive experiments. Experimental results consistently demonstrate that taking into account the spurious correlation can significantly improve the adversarial robustness. In addition, the conducted adaptive attack (Tramer et al., 2020) demonstrates that the proposed method is genuinely robust.

heading: 4.1 SETUPS

text:
Baseline methods. Our experiments are designed to demonstrate the necessity of considering the spurious correlation between labels Y and style variables S when developing robust models. Eq. 7 shows that if we set the hyperparameter γ = 0, and omit the spurious correlation between Y and S, the proposed method is equivalent to the adversarial training variant Madry (Madry et al., 2017). Thus, to demonstrate the necessity of considering the spurious correlation, we set γ = 0 and λ > 0 in Eq. 7, named ADA-M, and compare the adversarial robustness of ADA-M with that of Madry. In addition, the model capacity is often insufficient in adversarial training (Zhang et al., 2020b), so replacing one-hot labels of the first term in Eq. 7 with soft targets, i.e., the model prediction h(X;θ), can relieve the problem of insufficient model capacity 5. Considering the insufficient model capacity, we replace Y in the first term of Eq. 7 with the model prediction, and the derived method is called ADA-T. We find that ADA-T becomes the objective introduced in TRADES (Zhang et al., 2019) when we omit the spurious correlation, i.e., λ = 0, which shows that the proposed method also shares the same spirits to TRADES. Therefore, to demonstrate the importance of the spurious correlation between Y and S, we compare ADA-M and ADA-T with Madry and TRADES, respectively.
Evaluation metrics and training details. To evaluate the robustness for different methods, we compute the test accuracy on natural and adversarial examples with `∞-norm bounded perturbation generated by: FGSM (Goodfellow et al., 2014), PGD (Madry et al., 2017), and C&W (Carlini & Wagner, 2017) attacks. The robustness is evaluated on both the best checkpoint model suggested by (Rice et al., 2020) and the last checkpoint model used in (Madry et al., 2017), respectively. For MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017; Zhang et al., 2019). For CIFAR10 and CIFAR100, two architectures are employed: ResNet-18 (He et al., 2016) and WRN34-10 (Zagoruyko & Komodakis, 2016). The settings of attacks and hyper-parameters for training are the same as previous works, more details can be found in Appendix D.

heading: 4.2 ROBUSTNESS EVALUATION

text:
We evaluate the robustness of Madry, TRADES, and the proposed method on MNIST, CIFAR10, and CIFAR100 against various attacks (Goodfellow et al., 2014; Madry et al., 2017; Carlini & Wagner, 2017), which are widely used in the literature. We report the classification accuracy on MNIST in Table 1, where “Natural” denotes the accuracy on natural test images. We denote by PGD-40 the
5This is because knowledge distillation (Hinton et al., 2015) demonstrates that models with insufficient capacity prefer soft targets.
PGD attack with 40 iterations for generating adversarial examples, which also applies to the C&W attack. The results of ResNet-18 on CIFAR10 and CIFAR100 are illustrated in Table 2 and Table 3, respectively. The results of WRN-34-10 are in Appendix E. We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can significantly improve the adversarial robustness. To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Appendix F. Note that, the standard deviations of 5 runs are omitted because they hardly affect the results.
4.3 DISCUSSION
Consideration of gradient obfuscation. According to the criterion suggested by (Athalye et al., 2018), we exclude the potential effect of gradient obfuscation by showing the following phenomenons: a) the performance of our method on FSGM attack (59.17%) is better than iterative attacks PGD20 (51.82%) and C&W-20 (50.93%); b) the performance of our method on black-box PGD-20 (83.65%) and C&W-20 (83.57%) attacks is better than that on white-box attacks (51.82%); c) strong attacks cause lower accuracy than weak attacks, i.e., the accuracy on PGD-20 and PGD-100 are 51.82% and 48.05%, respectively. In addition, no gradient shattering operator is used in our method. All these results are evaluated on the CIFAR10 dataset using the last checkpoint of ResNet-18. Thus, according to the criterion suggested by (Athalye et al., 2018), the robustness improvement of the proposed method does not result from gradient obfuscation. Consideration of adaptive attack. According to
the adaptive attack criterion (Tramer et al., 2020), we augment the original objective function used in the PGD attack with the proposed adversarial distribution alignment loss to implement adaptive attacks, i.e., Eq. 11. Under the adaptive attack, the accuracy is 51.68%, while under the original PGD-20 attack is 51.82%, demonstrating that the proposed method is genuinely robust.
Consideration of AutoAttack. Following previous work (Pang et al., 2020; Wang et al., 2019), we verify the effectiveness of the proposed method on AutoAttack. According to the commonly used setting (Pang et al., 2020; Wang et al., 2019), we report robust accuracy of WRN-34-10 trained with CIFAR10 dataset on Auto-Attack, Madry: 49.58%, ADA-M: 51.56%, TRADES: 52.46%, ADA-
T: 54.09%, HE (Pang et al., 2020): 53.74%, AWP (Wu et al., 2020): 56.17%, and MART (Wang et al., 2019): 56.29%, where HE, AWP and MART are adversarial variants achieving state-of-theart performance. These results show that the proposed method can endow models with robustness comparable to the state-of-the-art performance.
Mitigating robust overfitting. The recent work (Rice et al., 2020) first studies the robust overfitting phenomenon. Robust overfitting means that further training will increase the robust training accuracy and test accuracy on natural data after a certain training epoch, but the robust test accuracy will decrease. The robust overfitting phenomenon of Madry (Madry et al., 2017) is depicted in Fig. 2. It can be seen that the robust test accuracy of Madry decreases to about 44%, while the best robust accuracy of Madry is 51.92%. In contrast, the proposed method drastically reduces the difference between the best robust accuracy 54.44% and the robust accuracy 50.93% of the last checkpoint.

heading: 5 RELATED WORK

text:
A large body of work has a constructive contribution to adversarial learning and causal reasoning. We only introduce the most related papers in this section due to the limited space.
Adversarial attack. Since the realization of the adversarial example phenomenon (Biggio et al., 2013; Szegedy et al., 2014), tons of adversarial attacks have been proposed (Moosavi-Dezfooli et al., 2016; Goodfellow et al., 2014; Carlini & Wagner, 2017; Dong et al., 2018; Tu et al., 2019; Madry et al., 2017; Croce & Hein, 2020). Among these attacks, FGSM Goodfellow et al. (2014), PGD attack (Madry et al., 2017), C&W attack (Carlini & Wagner, 2017), and Auto-Attack (Croce & Hein, 2020) are the most commonly used attacks for evaluating robustness.
Adversarial defense. The development of adversarial attacks promotes the progress of adversarial defense and detection. Recent work on improving adversarial robustness mainly falls into two categories: certified defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) and empirical defense and detection (Najafi et al., 2019; Carmon et al., 2019; Shafahi et al., 2019; Wong et al., 2020; Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020; Ma et al., 2018; Gao et al., 2021). Detailed discussions of these exciting works can be found in Appendix G.
Causal reasoning. The field of graphical causality, like machine learning, has a long history, see Pearl (2009); Schölkopf et al. (2021); Peters et al. (2017). One purpose of causal reasoning is to pursue the causal effect of interventions, contributing to achieving the desired objectives. Recent work shows the benefits of introducing causality into machine learning from various aspects (Zhang et al., 2020a; Mitrovic et al., 2020; Teshima et al., 2020; Tang et al., 2020; Sauer & Geiger, 2021). More details about relevant works can be found in Appendix H.

heading: 6 CONCLUSION

text: In this paper, we provide a novel causality viewpoint for understanding and mitigating adversarial vulnerability. Through constructing the causal graph of the adversarial data generation process and formalizing the intuition of adversarial attacks, we show that the spurious correlation between labels and style variables is important for understanding and mitigating adversarial vulnerability. Inspired by the observation, we propose the adversarial distribution alignment method, which takes the spurious correlation into account for robustness improvement. In addition, we find that the proposed method shares the same spirits to existing adversarial training variants. In future work, we will develop more effective algorithms to leverage or eliminate the spurious correlation between labels and style variables to further improve adversarial robustness. In addition, we will explore the uses of counterfactual statements to explain and mitigate the adversarial vulnerability. In sum, we make a first step towards employing causality to contribute to adversarial learning.

heading: 7 ETHICS STATEMENT

text: This paper does not raise any ethics concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.

heading: 8 REPRODUCIBILITY STATEMENT

text: To ensure the reproducibility of experimental results, we will provide a link for an anonymous repository of the source codes for this paper during the discussion phase.

heading: A DERIVATION OF DISTRIBUTION DIVERGENCE

text:
Considering that the adversarial distribution often cannot be obtained analytically, we need to relax the distribution divergence in Eq. 4, i.e., d (P (Y ∣X) , Pθ (Y ∣X̃)). Without loss of generality, the metric d in Eq. 4 can be realized as total variation distance (TVD). Thus, according to the definition of TVD, we have
d (P (Y ∣X) , Pθ (Y ∣X̃)) ≤ d (Pθ (Y ∣X̃) , Qθ (Y ∣X̃)) + d (P (Y ∣X) , Qθ (Y ∣X̃)) , (12) where Qθ(Y ∣X) is the conditional distribution specified by the classifier h (X;θ). According to the Pinsker inequality, i.e., d (P,Q) ≤ √ KL(P ∣∣Q)
2 , where KL is the Kullback-Leibler divergence,
we have an upper bound of Eq. 12:
d (P (Y ∣X) , Pθ (Y ∣X̃)) ≤
√ KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃))
2 +
√ KL (P (Y ∣X) ∣∣Qθ (Y ∣X̃))
2 .
(13)
Now, we can employ the upper bound Eq. 13 as a surrogate loss. In practice, we can replace √ KL(P ∣∣Q)
2 with KL (P ∣∣Q). The intuition is that the difference between
√ KL(P ∣∣Q)
2
and KL (P ∣∣Q) is relatively small when KL (P ∣∣Q) is not extremely large, so the replacement will not introduce much difference. Thus, we derive Eq. 5: KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) + γKL (P (Y ∣X) ∣∣Qθ (Y ∣X)) , where γ is a tunable hyperparameter. Here, we give an intuitive explanation of why optimizing Eq. 5 can make the adversarial distribution similar to the natural distribution. Considering that both the adversarial distribution Pθ (Y ∣X̃) and the anchor Qθ (Y ∣X̃) are parameterized by θ, so optimizing θ to minimize KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) can force the adversarial distribution be similar to the distribution specified by the classifier h (X; θ), i.e., Qθ (Y ∣X̃). That is, optimizing Qθ (Y ∣X̃) will also change the adversarial distribution Pθ (Y ∣X̃). In addition, minimizing the divergence KL (P (Y ∣X) ∣∣Qθ (Y ∣X̃)) can endow the classifier with the ability to provide a good prediction performance. Thus, minimizing KL (Pθ (Y ∣X̃) ∣∣Qθ (Y ∣X̃)) will make the adversarial distribution be similar with the natural distribution.
To verify the replacement will not introduce much difference, we evaluate the robustness of ADA-M and ADA-T on the CIFAR-10 dataset with two losses, i.e., Eq. 5 and Eq. 13. The results evaluated on best checkpoints are shown in Table 4.

heading: B CALCULATION OF THE EXPECTATION ON THE STYLE INFORMATION

text:
We provide details of calculating Eŝ(X̃)∼N (µ(X̃),σ2I)CE (g (ŝ (X̃) ;Wg) , Y ). We assume a normal distribution for the styles, i.e., ŝ (X̃) ∼ N (µ (X̃) , σ2I). According to the definition of the crossentropy loss, for a input pair (x, y) we have:
Eŝ(x)∼N (µ(x),σ2I)CE (g (ŝ (x) ;Wg) , y)
= Eŝ(x)∼N (µ(x),σ2I) log 1
P (Y = y∣g (ŝ (x) ;Wg)))
≤ log 1
Eŝ(x)∼N (µ(x),σ2I)P (Y = y∣g (ŝ (x) ;Wg)))
= log 1
Eŝ(x)∼N (µ(x),σ2I) e W⊤g,yŝ(x)
∑j e W⊤g,j ŝ(x)
= logEŝ(x)∼N (µ(x),σ2I) ∑ j
e (Wg,j−Wg,y)⊤ŝ(x)
= log∑ j
e (Wg,j−Wg,y)⊤ŝ(x)+ 12 (Wg,j−Wg,y) ⊤ σ 2 I(Wg,j−Wg,y)
= log ∑j eW
⊤ g,j ŝ(x)+σ
2 2 (Wg,j−Wg,y)⊤(Wg,j−Wg,y)
W⊤g,y ŝ(x)
≜ log 1 P ((Y = y∣g (ŝ(x);Wg)) ≜ CE (g (ŝ (x) ;Wg) , y) ,
(14)
where the inequality follows from the Jensen’s inequality: E log(X) ≤ logEX , the expectation is calculated by leveraging the moment-generating function:
EetX = etµ+ 1 2 σ 2 t 2
, X ∼ N (µ, σ2). (15) Note that, we define the function g (ŝ (x) ;Wg) for simplicity:
P ((Y = y∣g (ŝ(x);Wg)) ≜ W ⊤ g,y ŝ(x)
∑j=1 eW ⊤ g,j ŝ(x)+σ
2 2 (Wg,j−Wg,y)⊤(Wg,j−Wg,y)
. (16)
Besides seting g to a linear model, non-linear models, e.g., neural networks, can also be employed in practice. To verify the influence introduced by selecting different instantiation of model g, we compare different realizations of mode g, i.e., linear mapping and non-linear neural networks. The following results suggest that our method is relatively robust to the selection of model g. Note that, all these results are evaluated on the best checkpoint models trained on CIFAR-10 dataset.

heading: C RELATIONSHIP BETWEEN ORTHOGONALITY AND STATISTICAL INDEPENDENCE

text:
We give the proof for the following lemma in Sec. 3.3. Note that, we use R to present the learned representation of X , and replace X with R for simplicity.
Lemma 1. R ∈ Rd is the learned representation, where d is the number of dimension of R. Assume that R is a normal distribution with mean m and covariance matrix M . The content used for predicting labels, i.e., logits, is obtained by applying a linear functions to R, i.e., ĉ (R) =WcR, where Wc are parameters used for mapping R to logits. The style is modeled by a normal distribution, i.e., ŝ (R) = µ (R;Ws) + Σ (Y ) 1 2 n, where Ws presents parameters for modeling the mean of styles, and n is sampled from a standard normal distribution. Assume that µ (R;Ws) is a linear function, i.e., ŝ (R) = WsR + Σ (Y ) 1 2 n. Then, setting Ws as an instantiate of the orthogonal complement of Wc leads to statistical independence, i.e., ĉ (R) ⫫ ŝ (R). Here, ⫫ denotes the statistical independence, and we define ⟨a, b⟩M = ⟨a,Mb⟩ for a given semi-definite matrix M . The orthogonality A ⊥M B of two subspaces A and B is defined likewise.
Proof. Under the assumption in Lemma 1, setting Ws as an instantiate of the orthogonal complement of Wc, we have:
ker(Ws)⊥ ⊥M ker(Wc)⊥⟺ im(W⊤s ) ⊥M im(W⊤c )⟺ ⟨W⊤s a,W⊤c b⟩M = 0∀a, b ⟺ ⟨W⊤s a,MW⊤c b⟩ = 0∀a, b⟺WsMW⊤c = 0⟺ ERWs (R −m) (R −m)⊤W⊤c = 0 ⟺ ER,nWs (R + Σ 1 2n −m) (R −m)⊤W⊤c = 0⟺ Cov (ŝ (R) , ĉ (R)) = 0
⟺ ĉ (R) ⫫ ŝ (R) (17)

heading: D MORE DETAILS ABOUT EVALUATION METRICS AND TRAINING DETAILS

text:
Evaluation metrics. For MNIST dataset, we set the maximum perturbation bound = 0.3, perturbation step size η = 0.01, and the number of iterations K = 40 for PGD and C&W attacks, which keeps the same as (Zhang et al., 2019). Following (Rice et al., 2020), we set perturbation bound = 8/255, perturbation step size η = /10, and the number of iterations K = 20 for CIFAR-10 dataset.
Training details. For MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017; Zhang et al., 2019). Following (Zhang et al., 2019), the network is trained using SGD with 0.9 momentum for 50 epochs with an initial learning rate 0.01, and the batch size is set to 128. Hyperparameters used to craft adversarial examples for training are the same as those used for evaluation. These two networks share the same hyper-parameters: we use SGD with 0.9 momentum, weight decay 2 × 10−4, batch size 128, and an initial learning rate of 0.1. The maximum epoch is 120, and the learning rate is divided by 10 at epoch 60 and 90, respectively. To generate adversarial examples for training, we set the maximal perturbation = 8/255, the perturbation step size η = 2/255, and the number of iterationsK = 10, which is the same as (Rice et al., 2020). In all of our experiments β is set to 1.0. For ADA-M, λ is set to 1.0 and 0.5 for CIFAR-10 and CIFAR-100 datasets, respectively. For ADA-T, λ is set to 0.5 and 1.0 for CIFAR-10 and CIFAR-100 datasets, respectively.

heading: E EXPERIMENTS OF WRN-34-10 ON CIFAR-10

text:
In Table 6, we report the accuracy of WRN-34-10 (Zagoruyko & Komodakis, 2016) of Madry, TRADES, and the proposed method on CIFAR-10 against various attacks, i.e., FGSM, PGD, and
C&W attacks, which are widely used in the literature. Here, “Natural” denotes the accuracy of natural test images. We denote by PGD-20 the PGD attack with 20 iterations for generating adversarial examples, which also applies to the C&W attack. We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can significantly improve the adversarial robustness. Note that the standard deviations of 5 runs are omitted, because they hardly affect the results.

heading: F ABLATION STUDY

text:
We implicitly conducted ablation studies when designing Table 1, Table 2, and Table 3. To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Table 7. Comparing Madry, TRADES, and ADA-M, we find that introducing the second (t2) and the third term (t3) can improve the robustness and that the effect of these two terms is close. Similarly, comparing TRADES and ADA-T, we see that introducing the third term (t3) can further improve the robustness.
To further analyze the superiority of our method, we compare ADA-M with Madry to explore which kind of adversarial examples that ADA-M is more robust to. These analyses are based on the spurious perspective, as one main conclusion of our paper is that the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables. Specifically, we calculate the KL-divergence between KL (P (Y ∣h (x, s)) ∣∣P (Y ∣h (xadv, s))) for each input x, and divide samples into several (10 in our experiment) bins according to the KL-divergence. Then, we evaluate the robust accuracy of models trained with ADA-M and Madry in each bin sample, and results are depicted in Fig. 3. We can see that ADA-M is more robust to samples leading
to large KL-divergence than Madry. According to these empirical results, we can conclude that the proposed method is more robust to samples causing a significant difference between natural and adversarial distributions.
To further verify that ADA-T can outperform TRADES interms of both the natural and adversairal robustness, we compare the robust accuracy of ADA-T with that of TRADES on CIFAR-10 and CIFAR-100 dataset. The results evaluated on the best checkpoint models are shown in Table 8. We
can see that ADA-T can improve both the natural and the adversarial accuracy.
To explore the sensitivity of our method on hyperparameters, we evaluate our methods on CIFAR10 and CIFAR100 datasets. The results are organized in Table 9. Results in Table 9 demonstrate that
both ADA-M and ADA-T are relatively insensitive to the hyperparameters.

heading: G MORE DETAILS ABOUT ADVERSARIAL LEARNING

text:
Recent work on improving adversarial robustness mainly falls into two categories: certified defense and empirical methods.
Certified defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) aims to endow the model with provably adversarial robustness against norm-bounded perturbations. Although the certified defense strategy is promising, the empirical defense (Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019; Wang et al., 2019; Pang et al., 2020; Wong & Kolter, 2018; Xie et al., 2019; Yang et al., 2019), especially the adversarial training method (Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019), is currently the most effective strategy. Empirical defense firstly generates adversarial examples using a certain adversarial attack, then incorporates the generated adversarial examples into the training process.
Recently, various efforts (Najafi et al., 2019; Carmon et al., 2019; Shafahi et al., 2019; Wong et al., 2020; Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020) have been devoted to improving adversarial training. One line of work focuses on accelerating the training procedure (Shafahi et al., 2019; Wong et al., 2020). Another line of research (Najafi et al., 2019; Carmon et al., 2019) shows a promising direction that unlabeled training data can significantly mitigate the adversarial vulnerability. Lastly, recent work (Wang et al., 2019; Pang et al., 2020; Zhang et al., 2020b; Rice et al., 2020) provides an interesting direction where these methods rethink the adversarial training from different aspects, containing rethinking the misclassified examples (Wang et al.,
2019), rethinking the importance weight of each example (Zhang et al., 2020b) and rethinking the role of normalization (Pang et al., 2020) and basic training strategies (Rice et al., 2020). However, all these methods overlook the spurious correlation between labels and the style information.
Another related work is (Ilyas et al., 2019), which provides an interesting viewpoint, i.e., adversarial examples can be viewed as a human phenomenon because the model’s reliance on useful but not robust features leads to adversarial vulnerability. Our work gives a new causal perspective of adversarial vulnerability. Specifically, a) (Ilyas et al., 2019) found some features were useful but not robust, while our work explores the phenomenon’s fundamental cause and provides a clear explanation of why some features are useful but not robust: GivenX , labels Y are spuriously correlated with the style variables, so fitting the spurious correlation can predict labels. Thus, the style variables can be viewed as ‘features’; b) (Ilyas et al., 2019) claimed that adversarial examples could be viewed as a human phenomenon, while our work shows that adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon. Specifically, the adversarial vulnerability results from fitting the correlation between labels and style variables and failing to fit the causal relations, i.e., DNNs fail to extract content variables.

heading: H MORE DETAILS ABOUT CAUSAL REASONING

text:
The most relevant work is CAMA (Zhang et al., 2020a) that aims to improve the robustness of DNNs on unseen perturbation via explicitly modeling the perturbation from a causal view. The main difference between our method and CAMA is that we focus on the adversarial vulnerability while CAMA aims to improve the robustness of unseen perturbations. In addition, CAMA assumes a hard intervention on a latent variable. It promotes robustness via modeling the perturbation in the latent space. In this paper, we employ a soft intervention and propose to penalize DNNs when the adversarial distribution is different from the natural distribution. A recent work (Bühlmann, 2020) also aims to connect robust learning and causality, but the main focus of (Bühlmann, 2020) is on out-of-distribution generalization, which is different from adversarial learning.
Another related work is RELIC (Mitrovic et al., 2020), a regularizer used in self-supervised learning that uses the independence of mechanisms (Peters et al., 2017) and encourages DNNs to be invariant to different augmentations of the same instance. The self-supervised learning method (Mitrovic et al., 2020) also constructs a causal graph to model the data generation process, but the focus of RELIC is on the content invariant property, overlooking the importance of style information. Although (Sagawa et al., 2019) takes spurious correlations into account, (Sagawa et al., 2019) proposes using prior knowledge to group the training data to avoid the reliance on spurious correlations. Difference from (Sagawa et al., 2019), our method does not rely on prior knowledge. Another related work is CORE (Heinze-Deml & Meinshausen, 2021), a regularizer inspired by a causal graph is proposed to minimizing the variance of prediction and loss condition on label and ID information to mitigate the influence of domain shift. Different from (Heinze-Deml & Meinshausen, 2021), our method is designed to eliminate the difference between the natural and adversarial distributions.