summaries: This paper presents a method to detect adversarial examples by disentangling the representation. This is done by training autoencoders over both correct and incorrect class/semantic features. The authors claim that the proposed method outperformed other SOTA methods for adversarial detection and even works against adaptive adversarial attacks.  The paper propose a new defense mechanism against adversarial attacks based on an autoencoder architecture. The proposed architecture uses self-supervised learning to disentangle the semantic and class features, which enables the autoencoder to discriminate the clean and adversarial examples at test time. The approach was shown to outperforms other autoencoder approach in the experiments, when the attacker as access to only the victim model or both the victim model and autoencoder. The paper proposes a novel adversarial example detection strategy based on disentangled latent representations. The main idea is that latents are separated into class features and semantic features. Detection then uses the fact that adversarial samples preserve semantics while at the same time changing class according to some victim model. Training is being performed through a sophisticated combination of losses accounting for begin examples and counterexamples. Experimental results on detecting various kinds of attacks on popular datasets show clear improvements over previous methods. The paper also presents an evaluation of an adaptive attack. This paper proposes to detect adversary using learned disentangled auto-encoder. The model can disentangle the class feature and the semantic feature. Class features are easily changed by adversary, while the semantic features cannot. Existing Auto-Encoder cannot disentangle those two types of features, and also has strong generalization ability---high performance on both the adversary and the benign examples, making it hard to detect. Using the proposed disentangled AE, benign example can be reconstructed faithfully while adversary examples cannot. Thanks to the tailored AE, the paper state-of-the-art adversarial detection performance on MNIST, Fashion-MNIST, and CIFAR-10 in most cases.
abstractText: Deep learning models are known to be vulnerable to adversarial examples that are 1 elaborately designed for malicious purposes and are imperceptible to the human 2 perceptual system. Autoencoder, when trained solely over benign examples, has 3 been widely used for (self-supervised) adversarial detection based on the assump4 tion that adversarial examples yield larger reconstruction error. However, because 5 lacking adversarial examples in its training and the too strong generalization ability 6 of autoencoder, this assumption does not always hold true in practice. To alleviate 7 this problem, we explore to detect adversarial examples by disentangled represen8 tations of images under the autoencoder structure. By disentangling input images 9 as class features and semantic features, we train an autoencoder, assisted by a 10 discriminator network, over both correctly paired class/semantic features and in11 correctly paired class/semantic features to reconstruct benign and counterexamples. 12 This mimics the behavior of adversarial examples and can reduce the unnecessary 13 generalization ability of autoencoder. We compare our method with the state-of14 the-art self-supervised detection methods under different adversarial attacks and 15 different victim models (30 attack settings), and it exhibits better performance in 16 various measurements (AUC, FPR, TPR) for most attack settings. Ideally, AUC is 17 1 and our method achieves 0.99+ on CIFAR-10 for all attacks. Notably, different 18 from other Autoencoder-based detectors, our method can provide resistance to the 19 adaptive adversary. 20
references.authors: C. Szegedy W. Zaremba I. Sutskever J. Bruna D. Erhan I. Goodfellow R. Fergus
references.title: Intriguing 300 properties of neural networks
references.venue: ICLR, 2013. 301
references.authors: I.J. Goodfellow J. Shlens C. Szegedy
references.title: Explaining and harnessing adversarial examples
references.venue: ICLR, 302 2015. 303
references.authors: S.-M. Moosavi-Dezfooli A. Fawzi P. Frossard
references.title: DeepFool: A simple and accurate method to fool 304 deep neural networks
references.venue: CVPR, pp. 2574–2582, 2016. 305
references.authors: N. Carlini D. Wagner
references.title: Towards evaluating the robustness of neural networks
references.venue: Oakland, pp. 39–57, 306 2017. 307
references.authors: A. Madry A. Makelov L. Schmidt D. Tsipras A. Vladu
references.title: Towards deep learning models resistant to 308 adversarial attacks
references.venue: ICLR, 2018. 309
references.authors: K. Roth Y. Kilcher T. Hofmann
references.title: The odds are odd: A statistical test for detecting adversarial 310 examples
references.venue: ICML, pp. 5498–5507, 2019. 311
references.authors: K. Grosse P. Manoharan N. Papernot M. Backes P. McDaniel
references.title: On the (statistical) detection of 312 adversarial examples
references.venue: arXiv preprint arXiv:1702.06280, 2017. 313
references.authors: K. Lee K. Lee H. Lee J. Shin
references.title: A simple unified framework for detecting out-of-distribution samples 314 and adversarial attacks
references.venue: NeurIPS, pp. 7167–7177, 2018. 315
references.authors: D. Hendrycks K. Gimpel
references.title: Early methods for detecting adversarial images
references.venue: ICLR, 2016. 316
references.authors: N. Carlini D. Wagner
references.title: Adversarial examples are not easily detected: Bypassing ten detection methods
references.venue: 317 in CCS, pp. 3–14, 2017. 318
references.authors: J. Tian J. Zhou Y. Li J. Duan
references.title: Detecting adversarial examples from sensitivity inconsistency of 319 spatial-transform domain
references.venue: AAAI, 2021. 320
references.authors: D. Meng H. Chen
references.title: Magnet: A two-pronged defense against adversarial examples
references.venue: CCS, pp. 135– 321 147, 2017. 322
references.authors: C. Chen J. Liu Y. Xie Y.X. Ban C. Wu Y. Tao H. Song
references.title: Latent Regularized Generative Dual 323 Adversarial Network For Abnormal Detection
references.venue: IJCAI, (California), pp. 760–766, 2020. 324
references.authors: D. Gong L. Liu V. Le B. Saha M.R. Mansour S. Venkatesh A. v. d. Hengel
references.title: Memorizing normality 325 to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection
references.venue: CVPR, 326 pp. 1705–1714, 2019. 327
references.authors: G. Vacanti A. Van Looveren
references.title: Adversarial detection and correction by matching prediction distributions
references.venue: 328 arXiv preprint arXiv:2002.09364, 2020. 329
references.authors: B. Wójcik P. Morawiecki M. Śmieja T. Krzyżek P. Spurek J. Tabor
references.title: Adversarial examples detection 330 and analysis with layer-wise autoencoders
references.venue: arXiv preprint arXiv:2006.10013, 2020. 331
references.authors: Y. Qin N. Frosst S. Sabour C. Raffel G. Cottrell G. Hinton
references.title: Detecting and diagnosing adversarial 332 images with class-conditional capsule reconstructions
references.venue: ICLR, 2019. 333
references.authors: L. Schott J. Rauber M. Bethge W. Brendel
references.title: Towards the first adversarially robust neural network 334 model on MNIST
references.venue: ICLR, 2018. 335
references.authors: A. Kurakin I. Goodfellow S. Bengio
references.title: Adversarial machine learning at scale
references.venue: ICLR, 2017. 336
references.authors: N. Carlini D. Wagner
references.title: Adversarial examples are not easily detected: Bypassing ten detection methods
references.venue: 337 in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3–14, 2017. 338
references.authors: J. Lu T. Issaranon D. Forsyth
references.title: Safetynet: Detecting and rejecting adversarial examples robustly
references.venue: 339 ICCV, pp. 446–454, 2017. 340
references.authors: R. Feinman R.R. Curtin S. Shintre A.B. Gardner
references.title: Detecting adversarial samples from artifacts
references.venue: 341 arXiv preprint arXiv:1703.00410, 2017. 342
references.authors: X. Ma B. Li Y. Wang S. Erfani S. Wijewickrema M. Houle G. Schoenebeck D. Song J. Bailey
references.title: Characterizing adversarial subspaces using local intrinsic dimensionality,
references.authors: N. Carlini D. Wagner
references.title: Magnet and "efficient defenses against adversarial attacks" are not robust to 345 adversarial examples
references.venue: arXiv preprint arXiv:1711.08478, 2017. 346
references.authors: X. Chen Y. Duan R. Houthooft J. Schulman I. Sutskever P. Abbeel
references.title: Infogan: Interpretable 347 representation learning by information maximizing generative adversarial nets
references.venue: NeurIPS, pp. 2172– 348 2180, 2016. 349
references.authors: M. Sang W. Xia J.H. Hansen
references.title: Deaan: Disentangled embedding and adversarial adaptation network 350 for robust speaker representation learning
references.venue: arXiv preprint arXiv:2012.06896, 2020. 351 10
references.authors: L. Tran X. Yin X. Liu
references.title: Disentangled representation learning gan for pose-invariant face recognition
references.venue: 352 in CVPR, pp. 1415–1424, 2017. 353
references.authors: A. Ilyas S. Santurkar D. Tsipras L. Engstrom B. Tran A. Madry
references.title: Adversarial examples are not 354 bugs, they are features
references.venue: NeurIPS, pp. 125–136, 2019. 355
references.authors: C. Xiao B. Li J.-Y. Zhu W. He M. Liu D. Song
references.title: Generating adversarial examples with adversarial 356 networks
references.venue: IJCAI, pp. 119–128, 2018. 357
references.authors: Y. LeCun L. Bottou Y. Bengio P. Haffner
references.title: Gradient-based learning applied to document recognition
references.venue: 358 Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. 359
references.authors: H. Xiao K. Rasul R. Vollgraf
references.title: Fashion-MNIST: A novel image dataset for benchmarking machine 360 learning algorithms
references.venue: arXiv preprint arXiv:1708.07747, 2017. 361
references.authors: A. Krizhevsky G. Hinton
references.title: Learning multiple layers of features from tiny images
references.venue: 2009. 362
references.authors: J. Rauber W. Brendel M. Bethge
references.title: Foolbox: A python toolbox to benchmark the robustness of 363 machine learning models
references.venue: ICML, 2017. 364
sections.text: Deep learning models are known to be vulnerable to adversarial examples that are1 elaborately designed for malicious purposes and are imperceptible to the human2 perceptual system. Autoencoder, when trained solely over benign examples, has3 been widely used for (self-supervised) adversarial detection based on the assump-4 tion that adversarial examples yield larger reconstruction error. However, because5 lacking adversarial examples in its training and the too strong generalization ability6 of autoencoder, this assumption does not always hold true in practice. To alleviate7 this problem, we explore to detect adversarial examples by disentangled represen-8 tations of images under the autoencoder structure. By disentangling input images9 as class features and semantic features, we train an autoencoder, assisted by a10 discriminator network, over both correctly paired class/semantic features and in-11 correctly paired class/semantic features to reconstruct benign and counterexamples.12 This mimics the behavior of adversarial examples and can reduce the unnecessary13 generalization ability of autoencoder. We compare our method with the state-of-14 the-art self-supervised detection methods under different adversarial attacks and15 different victim models (30 attack settings), and it exhibits better performance in16 various measurements (AUC, FPR, TPR) for most attack settings. Ideally, AUC is17 1 and our method achieves 0.99+ on CIFAR-10 for all attacks. Notably, different18 from other Autoencoder-based detectors, our method can provide resistance to the19 adaptive adversary.20
  1 Introduction21
  In 2013, the seminal work [1] reported that, during model test time, deep neural networks can be22 easily fooled by adversarial attacks that add tiny perturbation to inputs. Since then, adversarial23 attacks and defenses have drawn significant research attention [2–9]. On the one hand, attackers are24 persistently developing new strategies to construct adversarial examples; on the other hand, defenders25 are struggling to cope with all existing and forthcoming attacks [10].26
  Most of the existing defense methods [6–9, 11] are trained with supervision, and these methods work27 well when defending against adversarial attacks they were originally trained for. However, it is widely28 regarded that supervised methods cannot generalize well to adversarial examples from (existing)29 unseen attacks, let alone examples from new attacks.30
  Self-supervised based defense, in comparison with supervised defense, requires only benign examples31 for its training. As a typical example, the works in [12, 13] utilize the encoder of an autoencoder32 (AE) to draw the manifold of benign examples and then the decoder network for reconstruction, as33 shown in Fig. 1(a). Since the manifold is learnt from benign examples only and the AE is trained to34 minimize Reconstruction Errors (RE) for benign examples, thus the encoding of adversarial example35 will likely be out-of-distribution and the associated RE is larger.36
  Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.
  It is soon realized that this is not always true because AE has very strong generalization ability [14]:37 examples with various kinds of small perturbations can be reconstructed with small RE. This is38 desirable if the perturbed examples are benign. However, adversarial examples are just specific39 perturbed versions of benign examples and the malicious perturbations can be also made very small40 in many attacks (i.e., now their encodings will reside in the light gray area of Fig. 1(a)). When41 this happens, all REs are mixed and it leads to high false negative or false positive rate (FNR/FPR)42 during detection. To refine the volume of the manifold drawn by AE and reduce its unnecessary43 generalization ability on adversarial examples, there exist a number of variants [15–18], as will be44 reviewed in detail in Sec. 2.2.45
  As a better solution to this problem, we propose a self-supervised disentangled representation based46 reconstruction (DRR) method to detect adversarial examples. DRR possesses the advantage of47 supervised defense, even though it does not have access to any adversarial examples in training. This48 is achieved through mimicking the behavior of adversarial examples by encoding and decoding a49 special class of examples (counterexamples in this work), which is the reconstruction of semantic50 feature from one example and class feature from an uncorrelated example. The rationale is based on51 the very fact of adversarial examples: they cause misclassification without changing semantics52 (contained in its benign counterparts).53
  With this fact, we disentangle, with the help of the victim model and an auxiliary encoder, the54 representation of images as two parts: class-dependent and class-independent (i.e., class feature55 and semantic feature). After disentangling, we then train a decoder to reconstruct benign examples56 by combining class/semantic features from the same benign image, as well as counterexamples by57 combining class/semantic features from different benign images (i.e., images with different labels).58 During detection, as shown in Fig. 1(b), DRR will faithfully disentangle any benign image to paired59 class/semantic features and its adversarial counterpart to unpaired class/semantic features, whose60 associated REs are significantly different as desired.61
  This paper makes the following contributions: 1) We use disentangled representation for adversarial62 detection, which makes it possible for the detector to mimic the behavior of adversarial examples in63 the self-supervised framework. 2) We design DRR via an AE structure, but it reduces the unnecessary64 generalization capability of AE on adversarial examples. 3) We achieve state-of-the-art adversarial65 detection performance on MNIST, Fashion-MNIST, and CIFAR-10 in most cases. Specifically, the66 Area Under Curve of Receiver Operation Characteristic (ideal AUC of ROC value is 1) is 0.99+ on67 CIRFAR-10.68
  2 Background and Related Works69 2.1 Constructing Adversarial Examples70
  The existence of adversarial examples in deep neural networks is first pointed out by [1], who find maliciously designed imperceptible perturbations can fool deep models to misclassify. Let F(·) be a general neural model and C(·) be the layers before softmax of the model, then evaluating the test example x is simply a softmax classification over the logits c = C(x), i.e., y = F(x) = softmax(c). The (untargeted) adversarial example xadv derived from x satisfies
  xadv = x+ δadv,F(xadv) 6= F(x), and ‖δadv‖ < , (1)
  where δadv is the adversarial perturbation, is the maximum magnitude of δadv under certain norm71 compliance. Commonly used norm could be L1, L2, and Linf , so we also focus on detecting72 adversarial examples bounded by these norms.73
  In the current literature, Fast Gradient Sign Method (FGSM) is the first widely used method in74 generating adversarial examples [2]. Basic Iterative Method (BIM) [19] and Projected Gradient75 Descent (PGD) [5] improve FGSM by iterating the building block of it according to different criteria76 to find the optimized perturbation. DeepFool [3] and CW [4] do not directly rely on gradient but they77 instead optimize (the norm of) δadv, which are usually more stealthy than FGSM and its variants.78
  Another line of research for constructing adversarial examples is called adaptive attacks [20]. Under79 an adaptive attack, the attacker also has white-box access to possible defense mechanisms, and80 his goal is to find an optimal δadv that solves Eq. (1) and circumvents the defense mechanisms81 simultaneously. All the attacks above can have their adaptive versions by considering different82 defenses.83
  2.2 Detecting Adversarial Examples84
  Adversarial detection is an effective way to prevent adversarial examples, and it can be classified85 as supervised and unsupervised methods considering whether adversarial examples are needed for86 training. For the case of supervised detectors, their detecting capability depends on how to capture87 the differences between adversarial and benign examples. Techniques range from studying statistical88 properties [6–8], training traditional machine learning classifiers [21, 22] and deep classifiers [9, 11,89 16, 23]. It is widely accepted that supervised methods cannot generalize well to adversarial examples90 produced by unseen attacks. For example, a supervised detector trained with PGD adversarial91 examples is likely to fail to detect examples produced by CW. However, it is also known that92 supervised detectors are robust to adaptive attacks, i.e., a detector trained with PGD will likely detect93 examples produced by the adaptive PGD attack [11, 23].94
  For the case of unsupervised detectors, their detecting capability depends on how to embed and95 represent benign examples to a different manifold (other than the natural spatiotemporal domain)96 thus adversarial perturbations will be magnified with embedding (without even seeing them at all).97 MagNet, proposed in [12], firstly uses an autoencoder to draw the manifold of benign examples, and98 it is regarded that the distance between the adversarial and benign examples are large via embedding99 and reconstruction.100
  However, the embedding manifold induced by autoencoder is not always desirable for detection,101 since autoencoder has too strong generalization capability [14]. Further to MagNet, the work [15]102 trains a variant of autoencoder by adding logits of the victim model into the loss function to refine103 the volume of the embedded manifold. The work [16] proposes to directly use parameters fixed104 victim model as the encoder, and the victim models’ logits as high-level representations/embedding.105 Different from the works above which treat the manifold of all benign examples as a whole, [17, 18]106 propose a class-conditional model to embed and reconstruct benign examples for even better refining107 the (embedding) manifold. However, it is reported in [17] that this method still only reacts to108 relatively large (adversarial) perturbation over simple datasets. Compared with supervised detectors,109 self-supervised detectors may generalize better to unseen attacks but they are vulnerable to adaptive110 attacks [12, 14, 16]. For example, it is shown in [24] that the detector of [12] fails to resist adaptive111 attacks at all.112
  2.3 Disentangled Representation113
  Disentangled representation is firstly advocated by InfoGAN in [25], which encourages the learning114 of interpretable and meaningful representations of inputs for manipulating specific features. The115 work in [26] disentangles speaker-related representation to synthesis voices of different speakers.116 The work in [27] uses disentangled representation to address pose discrepancy problem among face117 images.118
  This paper uses disentangled representations for adversarial detection. It is pointed out [28] that119 there are many features in an image, and in adversarial attack, not all features are equally easy to120 be manipulated. Specifically, the class an image belongs to, which does not depend on all semantic121 features it has, is a concrete example. In this concern, we disentangle representations of an image122
  into semantic feature and class feature. Semantic features can be easily manipulated by the attackers123 (through adversarial perturbation) but the perturbed features are similar to the original version (even124 after embedding if the perturbation is tiny enough). In contrast, class features cannot be directly125 manipulated (by attackers), but they are fragile to adversarial perturbation and robust to natural126 perturbation of the semantic features.127
  3 DRR for Adversarial Detection128
  This section presents the details of how to train DRR and use DRR to detect adversarial examples. For129 detection, as discussed earlier and shown in Fig. 1(b), an incoming test example x will be encoded130 and decoded as x′, then the RE ‖x′ − x‖2 is compared to a threshold value. If the RE is larger, then131 it is considered as adversarial; otherwise, it is not. Ideally, this threshold should be universal (on a132 given dataset) for all possible attacks (even for new attacks). However, for existing detectors, the133 threshold value is related to the used attack methods. This makes it hard to determine a universal134 threshold for existing detectors and to compare with them. For this reason, we use ROC curve of135 each attack strategy as the metric for evaluation in Sec. 4. But it is worth mentioning that it is easy to136 set a universal threshold for DRR for all the considered attacks.137
  We then move to how to train DRR. As depicted in Fig. 2, assisted by the parameter-fixed victim138 model V, DRR first extracts class and semantic features from one benign example xi to train the139 encoder En and decoder De. Then DRR takes the class feature from another benign example xj140 (j 6= i and xj’s label is different to xi) and combines it with the semantic feature from xi to mimic141 the behavior of adversarial examples. The two encoding-decoding processes are further enhanced142 with the discriminator D. The details of how to design the loss functions of the modular networks are143 presented in detail below.144
  First and foremost, we examine how to disentangle an input image x to semantic feature s and145 class feature c for our detection purpose. For s, which represents the semantics of x, we use a146 general encoder network En to derive it, i.e., s = En(x). This is because it is widely accepted that147 high-dimensional input x resides in a low-dimensional manifold, and it is a commonly used method148 in the area of representation learning. For c, which represents the class feature of x, we choose to use149 the logits from the victim model1 V, i.e., c = C(x). Aligned with the encoder-decoder structure and150 the discussion in Sec. 2.3, the rationale for this choice is two-sided: 1) Different from the one-hot151 encoding of the label of x, C(x) contains richer information for the subsequent decoding process; 2)152 Regardless of how the concrete adversarial example xadv is constructed, C(xadv) must be different153 enough from C(x) to induce the final erroneous classification.154
  With these features available, a decoder network De is then used to reconstruct x′ from s and c (of x), i.e., x′ = De(s, c). The natural requirement for En and De is that, for a generic image xi from the benign dataset, the reconstructed version x′i should be similar to xi. Moreover, we need to reserve the generalization capability over benign examples. For this purpose, an Gaussian noise vector ni, each of the i.i.d. component follows N (0, δ2), is used for data augmentation, i.e., x̂i = xi + ni. Thus, the
  1We hereinafter abuse the notation V in Fig. 2 to denote victim model without the final softmax layer as this will not cause ambiguity.
  associated loss is
  L1 = ExiMSE(xi, x̂′i), (2) where x̂′i = De (En(x̂i),C(x̂i)), MSE is the mean squared error and E is the expectation.155 Different from the normal AE based detection and its variants, we also require the class features, after encoding and decoding of xi or its noisy version x̂i, are still aligned. So, another loss function associated with xi is
  L2 = Exi [MSE(ci, c′i) +MSE(ĉi, ĉ′i)] , (3) where ci = C(xi), c′i = C(x ′ i), ĉi = C(x̂i) and ĉ ′ i = C(x̂ ′ i). It is worth mentioning that we use the156
  layer normalized version of ci/c′i and ĉi/ĉ ′ i to calculate this cost in the experiment to avoid numeric157 instability and ease the task of hyperparameter tuning. This finishes our discussion of how to train En158 and De with paired semantic feature and class feature from benign examples (the red box with solid159 line in Fig. 2).160
  We then move to the training of DRR with unpaired semantic feature and class feature, as depicted by161 the red box with dashed line in Fig. 2. As emphasized in Sec. 2.2, the drawback of AE based detection162 is AE generalizes too well and the refinement of the manifold drawn by AE is not always effective,163 especially on attacks that directly optimize the norm of the adversarial perturbation. The solution164 is now straightforward since we can mimic the behavior of adversarial examples by constructing165 counterexamples from unpaired semantic feature and class feature to better refine the manifold drawn166 by En and De.167
  For another benign example xj with j 6= i, we extract its class feature via cj = C(xj). To obtain a168 counterexample, the semantic feature si from xi and the class feature cj from xj are then passed to169 De to get x′i,j = De (En(xi),C(xj)). Recall that our original purpose is to refine the manifold drawn170 by the encoder and decoder, so the loss function here should satisfy the following two requirements:171
  • The decoded x′i,j should not converge to x′i (and thus xi) because convergence indicates that172 counterexamples and benign examples are mixed, which is contrary to our original detection173 purpose;174
  • The decoded x′i,j should not be far away from x′i, since out-of-distribution examples do not175 really improve the detection capability of autoencoder.176
  For these reasons, we choose to use a soft hinge function as the loss, i.e.,
  L3 = Exi ∑ j,j 6=i max(0, d− ∥∥x′i,j − x′i∥∥2)  , (4) where d is a hyperparameter used to control the farthest allowable distance between counter and177 benign examples. This soft hinge function (passively) meets the second requirements list above: its178 gradient equals 0 when
  ∥∥x′i,j − x′i∥∥2 > d.179 To further reduce the unnecessary generalization capability of the encoder-decoder network, we enforce class feature consistency over the counterexamples and their noisy versions. This is achieved by using the following loss function:
  L4 = Exi ∑ j,j 6=i MSE(cj , ĉ ′ j)  , (5) where ĉ′j = De(x̂ ′ i,j) with x̂ ′ i,j = x ′ i,j+nj (the component of nj also follows the Gaussian distribution180 N (0, δ2)).181 Inspired by [29], we treat the overall encoder-decoder (though assisted with the parameter-fixed victim model V) as a generator and then couple it with the last component of DRR, i.e., a discriminator network D. So (En+De) and D form the structure of generative adversarial networks (GAN), whose loss function is
  LGAN = Exi [logD(xi)] +Exi [log(1− D(De(En(xi),V(xi))))] +Exi,xj [log(1− D(De(En(xi),V(xj))))].
  The goal of this loss is to encourage the decoded examples, either benign or counterexamples, to be182 indistinguishable from the original input. It is worth mentioning that this GAN framework can be183 directly incorporated to all other encoder-decoder-like detectors [12, 13, 15–18], though they do not184 use a disentangled representation, to refine the manifold drawn by encoder-decoder networks.185
  In summary, the loss function to train the encoder En, the decoder De, and the discriminator D is
  Loss = 4∑
  i=1
  λiLi + λGANLGAN, (6)
  where λi (i = 1, · · · , 4) and λGAN control the relative importance of each loss function. En, De, and186 D are obtained by solving the minmax problem argminEn,De maxD.187
  The last observation we made is that even though the training of DRR mimics the behavior of188 adversarial examples, it does not really see any of them. And an empirical fact is that the victim model189 V is normally confident on benign examples. In contrast, it is not mandatory for V to be confident on190 adversarial examples (e.g., max(softmax(C(xadv))) = 0.2 << max(softmax(C(x))) = 0.9). This191 leads to trivial failure of DRR as DRR relies on class feature c = C(x) for reconstruction. As a192 remedy of this problem, we use the class feature sharpening trick here, i.e., c is updated as193
  c = { α · c− (α− 1) ·mean(c), if std(c) < σ; c, others,
  (7)
  where std is the standard deviation function, σ is the standard deviation of the training benign194 examples, and α, an empirical sharpening constant, is set to 3.195
  In this section, we assess the performance of DRR on three datasets, MNIST [30], Fashion-MNIST197 [31] and CIFAR-10 [32], by comparing it with other state-of-the-art detectors [12, 15, 16] against198 the adversarial attacks FGSM, BIM, PGD, DeepFool and CW (taken from Foolbox [33]) and the199 adaptive PGD under differnt norms (L1, L2 and Linf ). As a proof-of-concept, two representative200 networks, an 8-layer CNN and VGG-16 [34], are used as the victim models. These settings generate201 (24 + 6) attacks and (24× 5) + 6 defenses, so only the representative results are reported here due to202 space limit and the complete results (including the source code) are provided in the supplementary203 file. We use a GeForce RTX 2080 to conduct the experiments.204
  4.1 Settings205
  DRR: DRR has three modules: encoder En, decoder De and discriminator D. It is suffice to say that206 the encoder En and the discriminator D have the same architecture as the victim model except the last207 fully connected layer: for CIFAR-10, there are 128 neurons in the last layer of En; for MNIST and208 Fashion-MNIST, only 4 neurons are used. The architecture of the decoder De is simply a mirror of En.209 Following the literature [11, 23], the hyperparameters [λi]4i=1 and λGAN of the Loss are determined210 by a binary search in [10−3, 106], and d and δ are determined empirically. The full details are listed211 in Table 1.212
  Methods for comparison: As mentioned above, three kinds of self-supervised detectors [12, 15, 16]213 are used for comparison. The structure of the encoder, the decoder for all these three methods are the214 same as of DRR. These methods can be classified as hidden vector based reconstruction (HVR) and215 high-level representation based reconstruction (HLR).216
  Table 2: Test accuracy of the victim models.
  Data Attack Accuracy
  C IF
  A R
  10
  benign 0.869 BIM Linf = 0.01 0.195
  BIM L1 = 5 0.588 PGD Linf = 0.01 0.267 PGD L2 = 0.1 0.744 PGD L2 = 0.3 0.352
  FGSM Linf = 0.05 0.092 DeepFool Linf 0.054
  CW L2 0.001
  M N
  IS T
  benign 0.993 BIM Linf = 0.3 0.001 BIM L1 = 50 0.012 PGD Linf = 0.3 0.000 PGD L2 = 1 0.724 PGD L2 = 2 0.086
  FGSM Linf = 0.1 0.762 DeepFool Linf 0.078
  CW L2 0.000
  Fa sh
  io n
  benign 0.926 BIM Linf = 0.05 0.021
  BIM L1 = 20 0.300 PGD Linf = 0.05 0.008
  PGD L2 = 1 0.240 PGD L2 = 2 0.095 FGSM Linf = 0.05 0.327 DeepFool Linf 0.052
  CW L2 0.003
  For [12], it is based on optimizing the pixel-level reconstruction error from the hidden vector outputted217 by the encoder. We term it as HVR-P. For [15], it optimizes both pixel and logits reconstruction218 error and we term it as HVR-L. For [16], it takes the logits from the parameter-fixed victim model as219 a high-level representation of examples for decoding, and it has two variants, HLR-P and HLR-L,220 depending on whether the loss function includes the logits reconstruction error or not. We also221 incorporate and optimize the GAN loss when implementing all these detectors: HVR-P, HVR-L,222 HLR-P, and HLR-L. Note that our implementations perform slightly better than the original methods223 in [12, 15, 16], but this provides a fair environment for evaluating the effectiveness of disentangled224 representation.225
  4.2 Results Analyses226
  With the settings above, we first train victim models on the 3 datasets, and then generate adversarial227 examples with the 5 attacks (FGSM, BIM, PGD, DeepFool, and CW) under different norm compliance228 ( in Eq. (1)). All these test examples, either benign or adversarial, are passed to the detector for229 inspection.230
  Preparing test examples: The two victim models, VGG-16 and 8-layer CNN, are trained on CIFAR-231 10 and MNIST/Fashion MNIST, respectively. For each test example, we generate the corresponding232 adversarial versions with 5 attacks under different norm compliance, and the test accuracy is listed in233 Table 2. From this table, it is clear that the victim models achieve 0.869, 0.926, 0.993 classification234 accuracy over benign test examples. Also from this table, the attacks DeepFool and CW, which235 directly optimize , are generally stronger than the other three. By increasing , other attacks also236 obtains better attack result (i.e., worse accuracy), e.g., PGD with different L2 constraints in Table 2.237 However, a larger adversarial perturbation will make the attack get detected easier.238
  Visual inspection: We visually compare the reconstructed results of the benign and adversarial239 examples for all the detectors, some examples are shown in Fig. 3. Inspecting columns 3 and 4240 of Fig. 3(a) and (b), it is clear that DRR, HLR-P and HLR-L generally have bigger reconstruction241 errors over adversarial examples, which validates that logits (of the victim model) do help to reduce242 undesirable generalization capability of the detectors.243
  We further box-plot the reconstruction errors of all benign and adversarial examples from CIFAR-244 10, the result is shown in Fig. 4. It is clear from this figure, under all attacks of CIFAR-10, the245 reconstruction errors of DRR are clearly separable. The reconstruction errors of HLR-P and HLR-L246 are also generally separable, but it has a higher FNR or FPR than that of DRR (depending on the247 concrete threshold of RE). This validates that the disentangled semantic and class features can better248 refine the volume of the embedded manifold and prevent undesirable generalization. Moreoever, as249 shown in this figure, it is easy to determine a universal threshold for DRR to detect all considered250
  attacks over CIFAR-10 (and it is similar for the other two datasets), but it is hard to do so for HVR-P,251 HVR-L, HLR-P, and HLR-L.252
  ROC curve and AUC: With benign samples and adversarial examples of CIFAR-10, the ROC curves253 of all the detectors are depicted in Fig. 5 for qualitative analysis. From Fig. 5, it is very clear that254 DRR can retain a high true positive detection rate (TPR) while keeping the FPR rate low. If the size255 of the AUC of ROC curve is larger, the detector is better and the ideal size can reach up to 1.256
  We use AUC of ROC to quantitatively study the effectiveness of all detectors over CIFAR-10, as well257 as extending this metric to MNIST and Fashion-MNIST. The results are tabulated in Table 3. It is258 clear that, for CIFAR-10, the AUC of DRR is the largest over all different attacks, and the value is259 very close to the ideal case 1. For MNIST and Fashion-MNIST, DRR outperforms other detectors in260 most cases. For the cases that DRR is inferior to HLR-P/HLR-L, the gap is tiny and we regard their261 performances are comparable to each other.262
  4.3 Defensing Against Adaptive Adversarial Attack263
  To further evaluate the performance of DRR, we assume that the attacker can not only access the victim model but also knows all the details of the detector. Under this adaptive assumption, the attacker’s goal is to fool both the victim model and the detector. Following the most-widely used adaptive attack strategy [8, 11, 20, 23, 24], the attack now aims to solve
  min xadv
  αLRE(xadv)− LCE(xadv, y)
  s.t. ‖xadv − x‖p < , p ∈ {1, 2, inf} (8)
  where LRE and LCE are respectively the attack’s loss function for the detector (i.e., reconstruction264 error) and for the victim model (i.e., cross entropy), and y = F(x) is the true label of x and α is the265
  parameter to control the relative importance of the two loss functions. It is worth mentioning that the266 detectors [12,15,16] in comparison fail to resist the adaptive attack defined above2. The reason is, for267 other AE-based detectors, no matter high-level representation (i.e., logits) is used or not, they fail268 to capture true features of adversarial examples by certain. In contrast, the AE in DRR mimics the269 behavior of adversarial examples via disentangled representation.270
  We evaluate this adaptive attack strategy under the same setting of Table 1, and the results for271 CIFAR-10 and MNIST are depicted in Fig. 6. Observing the first row of Fig. 6, it is clear the accuracy272 of victim model under adaptive attacks is higher than that without our defense, a.k.a., it is harder273 for the attacker to succeed when the detector DRR is deployed. Moreover, if the attacker focuses274 more on circumventing the detector by increasing α of Eq. (8) (from 102 to 105), it will be harder for275 him to succeed in constructing real adversarial examples. However, if the attacker puts less focus on276 bypassing the detector, further experimental results verify that it makes him easier to get detected.277
  A similar trend can be also observed when implementing the adaptive attack on MNIST (the second278 row of Fig. 6). But from this figure, it is clear that now it is harder for the attacker to succeed in279 attacking, for example, the attacker only has about 10% attack success rate (under L1 and L2 norm)280 even with small α = 102 (less attention on circumventing the detector). We regard this is because the281 semantic feature of MNIST is simple, and the network architecture used by DRR (Table 1) captures282 this feature very well and the defense is stronger (than that of CIFAR-10 which has more complex283 semantic features). In this concern, it is reasonable to speculate that DRR can be extended to other284 complex datasets with an appropriate choice of model architecture to capture the complex semantic285 features within the dataset, which we leave as future work.286
  5 Conclusion287
  In this study, we propose to make use of disentangled representation for self-supervised adversarial288 examples detection. The proposed method DRR is based on the very nature of adversarial examples:289 misleading classification results without changing images’ semantics (a lot). With disentangled290 class and semantic features, this nature inspires us to construct counterexamples to better guide291 the training of DRR. Compared with previous self-supervised detectors, DRR generally performs292 better under various measurements over different datasets and different adversarial attack methods.293 Not surprisingly, compared with other AE-based adversarial detectors, DRR is also more robust to294 adaptive adversaries. This makes DRR a promising candidate, when combined with other proactive295 strategies, for the defense of adversarial attacks. Moreover, disentangled representation is a ubiquitous296 property for many other data formats, including natural language, voice, and video signals, so DRR297 may be extended easily to provide adversarial defense to other domains.298
  2We note that [15] considered a weaker adaptive attack strategy (maximize the cross entropy after AE reconstruction but not directly optimize reconstruction error) and HVR-L is robust to the weaker adaptive notion.
  References299 [1] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing300 properties of neural networks,” ICLR, 2013.301
  [2] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” ICLR,302 2015.303
  [3] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “DeepFool: A simple and accurate method to fool304 deep neural networks,” in CVPR, pp. 2574–2582, 2016.305
  [4] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in Oakland, pp. 39–57,306 2017.307
  [5] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to308 adversarial attacks,” in ICLR, 2018.309
  [6] K. Roth, Y. Kilcher, and T. Hofmann, “The odds are odd: A statistical test for detecting adversarial310 examples,” in ICML, pp. 5498–5507, 2019.311
  [7] K. Grosse, P. Manoharan, N. Papernot, M. Backes, and P. McDaniel, “On the (statistical) detection of312 adversarial examples,” arXiv preprint arXiv:1702.06280, 2017.313
  [8] K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples314 and adversarial attacks,” in NeurIPS, pp. 7167–7177, 2018.315
  [9] D. Hendrycks and K. Gimpel, “Early methods for detecting adversarial images,” ICLR, 2016.316
  [10] N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,”317 in CCS, pp. 3–14, 2017.318
  [11] J. Tian, J. Zhou, Y. Li, and J. Duan, “Detecting adversarial examples from sensitivity inconsistency of319 spatial-transform domain,” in AAAI, 2021.320
  [12] D. Meng and H. Chen, “Magnet: A two-pronged defense against adversarial examples,” in CCS, pp. 135–321 147, 2017.322
  [13] C. Chen, J. Liu, Y. Xie, Y. X. Ban, C. Wu, Y. Tao, and H. Song, “Latent Regularized Generative Dual323 Adversarial Network For Abnormal Detection,” in IJCAI, (California), pp. 760–766, 2020.324
  [14] D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. v. d. Hengel, “Memorizing normality325 to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection,” in CVPR,326 pp. 1705–1714, 2019.327
  [15] G. Vacanti and A. Van Looveren, “Adversarial detection and correction by matching prediction distributions,”328 arXiv preprint arXiv:2002.09364, 2020.329
  [16] B. Wójcik, P. Morawiecki, M. Śmieja, T. Krzyżek, P. Spurek, and J. Tabor, “Adversarial examples detection330 and analysis with layer-wise autoencoders,” arXiv preprint arXiv:2006.10013, 2020.331
  [17] Y. Qin, N. Frosst, S. Sabour, C. Raffel, G. Cottrell, and G. Hinton, “Detecting and diagnosing adversarial332 images with class-conditional capsule reconstructions,” in ICLR, 2019.333
  [18] L. Schott, J. Rauber, M. Bethge, and W. Brendel, “Towards the first adversarially robust neural network334 model on MNIST,” in ICLR, 2018.335
  [19] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” ICLR, 2017.336
  [20] N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,”337 in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3–14, 2017.338
  [21] J. Lu, T. Issaranon, and D. Forsyth, “Safetynet: Detecting and rejecting adversarial examples robustly,” in339 ICCV, pp. 446–454, 2017.340
  [22] R. Feinman, R. R. Curtin, S. Shintre, and A. B. Gardner, “Detecting adversarial samples from artifacts,”341 arXiv preprint arXiv:1703.00410, 2017.342
  [23] X. Ma, B. Li, Y. Wang, S. Erfani, S. Wijewickrema, M. Houle, G. Schoenebeck, D. Song, and J. Bailey,343 “Characterizing adversarial subspaces using local intrinsic dimensionality,” ICLR, 2018.344
  [24] N. Carlini and D. Wagner, “Magnet and "efficient defenses against adversarial attacks" are not robust to345 adversarial examples,” arXiv preprint arXiv:1711.08478, 2017.346
  [25] X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable347 representation learning by information maximizing generative adversarial nets,” in NeurIPS, pp. 2172–348 2180, 2016.349
  [26] M. Sang, W. Xia, and J. H. Hansen, “Deaan: Disentangled embedding and adversarial adaptation network350 for robust speaker representation learning,” arXiv preprint arXiv:2012.06896, 2020.351
  [27] L. Tran, X. Yin, and X. Liu, “Disentangled representation learning gan for pose-invariant face recognition,”352 in CVPR, pp. 1415–1424, 2017.353
  [28] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not354 bugs, they are features,” in NeurIPS, pp. 125–136, 2019.355
  [29] C. Xiao, B. Li, J.-Y. Zhu, W. He, M. Liu, and D. Song, “Generating adversarial examples with adversarial356 networks,” in IJCAI, pp. 119–128, 2018.357
  [30] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”358 Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.359
  [31] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-MNIST: A novel image dataset for benchmarking machine360 learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.361
  [32] A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009.362
  [33] J. Rauber, W. Brendel, and M. Bethge, “Foolbox: A python toolbox to benchmark the robustness of363 machine learning models,” in ICML, 2017.364
  [34] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”365 ICLR, 2015.366
  Checklist367 1. For all authors...368
  (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu-369 tions and scope? [Yes]370
  (b) Did you describe the limitations of your work? [Yes]371 (c) Did you discuss any potential negative societal impacts of your work? [Yes]372 (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]373
  2. If you are including theoretical results...374
  (a) Did you state the full set of assumptions of all theoretical results? [N/A]375 (b) Did you include complete proofs of all theoretical results? [N/A]376
  3. If you ran experiments...377
  (a) Did you include the code, data, and instructions needed to reproduce the main experimental378 results (either in the supplemental material or as a URL)? [Yes]379
  (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?380 [Yes] Please refer to Sec. 4 for the details.381
  (c) Did you report error bars (e.g., with respect to the random seed after running experiments382 multiple times)? [Yes]383
  (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,384 internal cluster, or cloud provider)? [Yes] Please refer to Sec. 4 for the details.385
  4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...386
  (a) If your work uses existing assets, did you cite the creators? [Yes]387 (b) Did you mention the license of the assets? [Yes]388 (c) Did you include any new assets either in the supplemental material or as a URL? [Yes] Code is389 available in the supplemental material.390 (d) Did you discuss whether and how consent was obtained from people whose data you’re us-391 ing/curating? [N/A]392 (e) Did you discuss whether the data you are using/curating contains personally identifiable informa-393 tion or offensive content? [N/A]394
  5. If you used crowdsourcing or conducted research with human subjects...395
  (a) Did you include the full text of instructions given to participants and screenshots, if applicable?396 [N/A]397
  (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB)398 approvals, if applicable? [N/A]399
  (c) Did you include the estimated hourly wage paid to participants and the total amount spent on400 participant compensation? [N/A]401
title: Self-Supervised Adversarial Example Detection by Disentangled Representation