summaries: This paper proposes a loss function for training the base classifier for randomized smoothed classifiers. Specifically, the loss distinguishes between training examples with high prediction confidence and those with low confidence. Several empirical tricks are applied to design the loss function to optimize the performance. Experiments on MNIST and CIFAR-10 show that the proposed training method is superior to existing state-of-the-art randomized smoothed classifiers, especially when the radius r is large. This paper proposed new loss functions during the training of classifiers for certified robustness via randomized smoothing. The new loss function treat samples with different confidence level differently. The main idea is to prioritize samples with high confidence because it provides more additional certified radius when its confidence grows. This paper proposes CAT-RS which combines two novel losses for training base classifiers for randomized smoothing. The two novel losses aim at preserving clean accuracy for hard samples and improving the certified radius for easy samples. In term of ACR (average certified radius), the method achieves state-of-the-art on MNIST and CIFAR-10. The paper studies certified robustness via randomized smoothing (RS). RS has a fundamental accuracy and robustness tradeoff. The authors aim to enhance such tradeoff through a sample-wise control of robustness over the training samples. In particular, the authors investigate the correspondence between robustness and prediction confidence of smoothed classifiers and design a new loss function. The proposed method is evaluated on MNIST and CIFAR10.  
abstractText: Any classifier can be “smoothed out” under Gaussian noise to build a new classifier that is provably robust to `2-adversarial perturbations, viz., by averaging its predictions over the noise, namely via randomized smoothing. Under the smoothed classifiers, the fundamental trade-off between accuracy and (adversarial) robustness has been well evidenced in the literature: i.e., increasing the robustness of a classifier for an input can be at the expense of decreased accuracy for some other inputs. In this paper, we propose a simple training method leveraging this trade-off for obtaining more robust smoothed classifiers, in particular, through a sample-wise control of robustness over the training samples. We enable this control feasible by investigating the correspondence between robustness and prediction confidence of smoothed classifiers: specifically, we propose to use the “accuracy under Gaussian noise” as an easy-to-compute proxy of adversarial robustness for each input. We differentiate the training objective depending on this proxy to filter out samples that are unlikely to benefit from the worst-case (adversarial) objective. Our experiments following the standard benchmarks consistently show that the proposed method, despite its simplicity, exhibits improved certified robustness upon existing state-of-the-art training methods.
references.authors: Dario Amodei Chris Olah Jacob Steinhardt Paul Christiano John Schulman Dan Mané
references.title: Concrete problems in AI safety
references.venue: arXiv preprint arXiv:1606.06565,
references.authors: Anish Athalye Nicholas Carlini David Wagner
references.title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
references.venue: In Proceedings of the 35th International Conference on Machine Learning,
references.authors: Anand Bhattad Min Jin Chong Kaizhao Liang Bo Li D.A. Forsyth
references.title: Unrestricted adversarial examples via semantic manipulation
references.venue: In International Conference on Learning Representations,
references.authors: dlish Alec Radford Ilya Sutskever Dario Amodei
references.title: Language models are few-shot learners
references.venue: Advances in Neural Information Processing Systems,
references.authors: Nicholas Carlini David Wagner
references.title: Towards evaluating the robustness of neural networks
references.venue: In IEEE Symposium on Security and Privacy,
references.authors: Nicholas Carlini Anish Athalye Nicolas Papernot Wieland Brendel Jonas Rauber Dimitris Tsipras Ian Goodfellow Aleksander Madry
references.title: On evaluating adversarial robustness
references.authors: Rich Caruana Yin Lou Johannes Gehrke Paul Koch Marc Sturm Noemie Elhadad
references.title: Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission
references.venue: In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,
references.authors: Ting Chen Simon Kornblith Mohammad Norouzi Geoffrey Hinton
references.title: A simple framework for contrastive learning of visual representations
references.venue: Proceedings of the 37th International Conference on Machine Learning,
references.authors: Jeremy Cohen Elan Rosenfeld Zico Kolter
references.title: Certified adversarial robustness via randomized smoothing
references.venue: In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning,
references.authors: Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger
references.title: On calibration of modern neural networks
references.venue: Proceedings of the 34th International Conference on Machine Learning,
references.authors: Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
references.title: Deep residual learning for image recognition
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang Evan Dorundo Rahul Desai Tyler Zhu Samyak Parajuli Mike Guo
references.title: The many faces of robustness: A critical analysis of out-of-distribution generalization
references.venue: arXiv preprint arXiv:2006.16241,
references.authors: Jongheon Jeong Jinwoo Shin
references.title: Consistency regularization for certified robustness of smoothed classifiers
references.venue: Advances in Neural Information Processing Systems,
references.authors: Jongheon Jeong Sejun Park Minkyu Kim Heung-Chang Lee Doguk Kim Jinwoo Shin
references.title: SmoothMix: Training confidence-calibrated smoothed classifiers for certified adversarial robustness
references.venue: In ICML 2021 Workshop on Adversarial Machine Learning,
references.authors: Alex Krizhevsky
references.title: Learning multiple layers of features from tiny images
references.venue: Technical report,
references.authors: Y. LeCun L. Bottou Y. Bengio P. Haffner
references.title: Gradient-based learning applied to document recognition
references.venue: Proceedings of the IEEE,
references.authors: Mathias Lecuyer Vaggelis Atlidakis Roxana Geambasu Daniel Hsu Suman Jana
references.title: Certified robustness to adversarial examples with differential privacy
references.venue: IEEE Symposium on Security and Privacy (SP),
references.authors: Kimin Lee Honglak Lee Kibok Lee Jinwoo Shin
references.title: Training confidence-calibrated classifiers for detecting out-of-distribution samples
references.venue: In International Conference on Learning Representations,
references.authors: Bai Li Changyou Chen Wenlin Wang Lawrence Carin
references.title: Certified adversarial robustness with additive noise
references.venue: In Advances in Neural Information Processing Systems
references.authors: Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu
references.title: Towards deep learning models resistant to adversarial attacks
references.venue: In International Conference on Learning Representations,
references.authors: Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi Pascal Frossard
references.title: DeepFool: a simple and accurate method to fool deep neural networks
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Hadi Salman Jerry Li Ilya Razenshteyn Pengchuan Zhang Huan Zhang Sebastien Bubeck Greg Yang
references.title: Provably robust deep learning via adversarially trained smoothed classifiers
references.venue: In Advances in Neural Information Processing Systems
references.authors: Hadi Salman Andrew Ilyas Logan Engstrom Ashish Kapoor Aleksander Madry
references.title: Do adversarially robust imagenet models transfer better
references.venue: In ArXiv preprint arXiv:2007.08489,
references.authors: David Silver Julian Schrittwieser Karen Simonyan Ioannis Antonoglou Aja Huang Arthur Guez Thomas Hubert Lucas Baker Matthew Lai Adrian Bolton
references.title: Mastering the game of go without human knowledge
references.authors: Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow Rob Fergus
references.title: Intriguing properties of neural networks
references.venue: In International Conference on Learning Representations,
references.authors: Florian Tramer Nicholas Carlini Wieland Brendel Aleksander Madry
references.title: On adaptive attacks to adversarial example defenses
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner Aleksander Madry
references.title: Robustness may be at odds with accuracy
references.venue: In International Conference on Learning Representations,
references.authors: Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser Illia Polosukhin
references.title: Attention is all you need
references.venue: Advances in Neural Information Processing Systems,
references.authors: Oriol Vinyals Igor Babuschkin Wojciech M Czarnecki Michaël Mathieu Andrew Dudzik Junyoung Chung David H Choi Richard Powell Timo Ewalds Petko Georgiev
references.title: Grandmaster level in starcraft ii using multi-agent reinforcement learning
references.authors: Yisen Wang Difan Zou Jinfeng Yi James Bailey Xingjun Ma Quanquan Gu
references.title: Improving adversarial robustness requires revisiting misclassified examples
references.venue: In International Conference on Learning Representations,
references.authors: Eric Wong Zico Kolter
references.title: Provable defenses against adversarial examples via the convex outer adversarial polytope
references.venue: Proceedings of the 35th International Conference on Machine Learning,
references.authors: Dongxian Wu Shu-Tao Xia Yisen Wang
references.title: Adversarial weight perturbation helps robust generalization
references.venue: Advances in Neural Information Processing Systems,
references.authors: Kai Y. Xiao Vincent Tjeng Nur Muhammad (Mahi) Shafiullah Aleksander Madry
references.title: Training for faster adversarial robustness verification via inducing ReLU stability
references.venue: In International Conference on Learning Representations,
references.authors: Ekim Yurtsever Jacob Lambert Alexander Carballo Kazuya Takeda
references.title: A survey of autonomous driving: Common practices and emerging technologies
references.venue: IEEE Access,
references.authors: Runtian Zhai Chen Dan Di He Huan Zhang Boqing Gong Pradeep Ravikumar Cho-Jui Hsieh Liwei Wang
references.title: MACER: Attack-free and scalable robust training via maximizing certified radius
references.venue: In International Conference on Learning Representations,
references.authors: Hongyang Zhang Yaodong Yu Jiantao Jiao Eric Xing Laurent El Ghaoui Michael Jordan
references.title: Theoretically principled trade-off between robustness and accuracy
references.venue: In Proceedings of the 36th International Conference on Machine Learning,
references.authors: Hongyi Zhang Moustapha Cisse Yann N. Dauphin David Lopez-Paz
references.title: mixup: Beyond empirical risk minimization
references.venue: In International Conference on Learning Representations,
references.authors: Huan Zhang Hongge Chen Chaowei Xiao Sven Gowal Robert Stanforth Bo Li Duane Boning Cho-Jui Hsieh
references.title: Towards stable and efficient training of verifiably robust neural networks
references.venue: In International Conference on Learning Representations,
references.authors: Jingfeng Zhang Xilie Xu Bo Han Gang Niu Lizhen Cui Masashi Sugiyama Mohan Kankanhalli
references.title: Attacks which do not kill training make adversarial learning stronger
references.venue: Proceedings of the 37th International Conference on Machine Learning,
references.authors: 2019 Salman et al 2020 Zhai et al Jeong 2020 Shin
references.title: 2021). We also normalize the images in pixel-wise by the mean and the standard deviation calculated from the training set. The full dataset can be downloaded at https: //www.cs.toronto.edu/ ̃kriz/cifar.html
references.venue: HYPERPARAMETERS
references.authors: Zhai
references.title: When σ = 1.0, for the training to succeed, we set λ = 6.0. For the CIFAR-10 experiments, we follow the original configurations used by Zhai et al. (2020). We set k = 16, γ = 8.0, and β = 16.0. λ is set to be 12.0 and 4.0 for σ = 0.25 and 0.5, respectively. For σ = 1.0, the training starts with λ = 0 until the first learning rate decay and we set λ = 12.0 thereafter
sections.heading: 1 INTRODUCTION
sections.text: Despite these tremendous advances in deep neural networks for a variety of tasks towards artificial intelligence, e.g., visual recognition (He et al., 2016; Chen et al., 2020), natural language processing (Vaswani et al., 2017; Brown et al., 2020), and reinforcement learning (Silver et al., 2017; Vinyals et al., 2019), the broad existence of adversarial examples (Szegedy et al., 2014) is still one of the most significant aspects that reveals the gap between machine learning systems and humans: for a given input x (e.g., an image) to a classifier f , say a neural network, f often permits a perturbation δ that completely flips the prediction f(x+ δ), while δ is too small to change the semantic in x. In response to this vulnerability, there have been significant efforts in building robust neural network based classifiers against adversarial examples, either in forms of empirical defenses (Athalye et al., 2018; Carlini et al., 2019; Tramer et al., 2020), which are largely based on adversarial training (Madry et al., 2018; Zhang et al., 2019; Wang et al., 2020; Zhang et al., 2020b; Wu et al., 2020), or certified defenses (Wong & Kolter, 2018; Xiao et al., 2019; Cohen et al., 2019; Zhang et al., 2020a), depending on whether the robustness claim can be theoretically guaranteed or not.
  Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019), our focus in this paper, is currently a prominent approach in the context of certified defense, thanks to its scalability to arbitrary neural network architectures while previous methods have been mostly limited in network sizes or require strong assumptions on their architectures: specifically, for a given classifier f , it constructs a new classifier f̂ , where f̂(x) is defined to be the class that f(x + δ) outputs most likely over δ ∼ N (0, σ2I), i.e., the Gaussian noise. Then, it is shown by Lecuyer et al. (2019) that f̂ is certifiably robust in `2-norm, and Cohen et al. (2019) further tightened the `2-robustness guarantee which is currently considered as the state-of-the-art in certified defense.
  However, even with recent methods for adversarial defense, including randomized smoothing, the trade-off between robustness and accuracy (Tsipras et al., 2019; Zhang et al., 2019) has been well evidenced, i.e., increasing the robustness for a specific input can be at the expense of decreased accuracy for other inputs. For instance, with the current best practices, Salman et al. (2020) report
  that the accuracy of ResNet-50 on ImageNet degrades, e.g., 75.8%→ 63.9%, by an `∞-adversarial training, i.e., optimizing the classifier to ensure robustness at all the given training samples around an `∞-ball (of size 4255 ). In addition, Zhang et al. (2019) have shown that the (empirical) robustness of a classifier can be further boosted in training by paying more expense in accuracy. A similar trend can be also observed with certified defenses, e.g., randomized smoothing, as the clean accuracy of smoothed classifiers are usually less than those one can obtain from the standard training on the same architecture (Cohen et al., 2019).
  Contribution. In this paper, we develop a novel training method for randomized smoothing, coined Confidence-Aware Training for Randomized Smoothing (CAT-RS), which incorporates a sample-wise control of target robustness on-the-fly motivated by the accuracy-robustness trade-off in smoothed classifiers. Intuitively, a natural approach one can consider in response to the trade-off in robust training is to appropriately lower the robustness requirement for “hard-to-classify” samples while maintaining those for the remaining (“easier”) samples: here, the challenges are (a) which samples we should choose for it during training and (b) how to control their target robustness.
  To implement this idea, we focus on the direct correspondence from prediction confidence to adversarial robustness that smoothed classifiers offer: due to its local-Lipschitzness (Salman et al., 2019), achieving a high confidence at x from a smoothed classifier also implies a high (certified) robustness at x. Inspired by this, we propose to use the sample-wise confidence (of smoothed classifiers) as an efficient proxy of the certified robustness, and defines two new losses, namely bottom-K and worstcase Gaussian training, each of those targets different levels of confidence so that the overall training can be more informed sample-wise for better robustness by preventing low-confident samples from being enforced to increase their robustness.
  We verify the effectiveness of our proposed method through an extensive comparison with existing robust training methods for smoothed classifiers, including the state-of-the-arts, on a wide range of established benchmarks on MNIST and CIFAR-10 datasets. Our experimental results constantly show that the proposed method can significantly improve the previous state-of-the-art results on certified robustness achievable from a given neural network architecture, by (a) maximizing the robust radii of high-confidence samples while (b) reducing the risk of deteriorating the accuracy at low-confidence samples. Our extensive ablation study further confirms that each of both proposed components has an individual effect on improving certified robustness, and can effectively control the accuracy-robustness trade-off with the hyperparameter between the two proposed losses.
sections.heading: 2 PRELIMINARIES
sections.text: Adversarial robustness. Consider a labeled dataset D = {(xi, yi)}ni=1 from a certain distribution P , where x ∈ Rd and y ∈ Y := {1, · · · ,K}, which forms a classification problem with K classes. Let f : Rd → Y be a classifier. Notice that f is a discrete and non-differentiable, so that one can additionally consider a differentiable F : Rd → ∆K−1 to allow a gradient-based optimization assuming f(x) := arg maxk∈Y Fk(x), where ∆
  K−1 is probability simplex in RK . The standard framework of empirical risk minimization to optimize f assumes that the samples in D are i.i.d. from P and expect f to perform well given that the future samples also follow the i.i.d. assumption.
  However, in the context of adversarial robustness (and for other notions of robustness as well), the i.i.d. assumption on the future samples does not hold anymore: instead, it additionally assumes that the samples can be arbitrarily perturbed up to a certain restriction, e.g., a bounded `2-ball, and focuses on the worst-case performance over the perturbed samples. One possible way to quantify this scenario is to consider the average minimum-distance of adversarial perturbation (MoosaviDezfooli et al., 2016; Carlini & Wagner, 2017; Carlini et al., 2019), namely:
  R(f ;P ) := E(x,y)∼P [
  min f(x′)6=y
  ||x′ − x||2 ] . (1)
  In this respect, we aim to find f that maximizes R(f ;P ) while maintaining the performance on P .
  Randomized smoothing. The essential challenge in achieving adversarial robustness in neural networks, however, stems from that directly evaluating (1) (and further optimizing it) is usually computationally infeasible, e.g., under the standard practice that F is modeled by a complex, highdimensional neural network. Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019)
  bypasses this difficulty by constructing a new classifier f̂ from f instead of letting f to directly model the robustness: specifically, it transforms the base classifier f with a certain smoothing measure, where in this paper we focus on the case of Gaussian distributions N (0, σ2I):
  f̂(x) := arg max c∈Y Pδ∼N (0,σ2I) (f(x+ δ) = c) . (2)
  Then, the robustness of f̂ at (x, y), namely R(f̂ ;x, y), can be explicitly lower-bounded in terms of the certified radius R(f̂ , x, y), e.g., Cohen et al. (2019) showed that the following bound holds which is tight for `2-adversary, e.g., it is the optimal for linear classifiers:
  R(f̂ ;x, y) ≥ σ · Φ−1(pf (x, y)) =: R(f̂ , x, y) (3) where pf (x, y) := Pδ(f(x+ δ) = y), (4)
  provided that f̂(x) = y, otherwise R(f̂ ;x, y) := 0.1 Here, we remark that the formula for certified radius (3) is essentially a function of pf (4), which represents the prediction confidence of f̂ at x, or equivalently, the accuracy of f(x + δ) over δ ∼ N (0, σ2I). In other words, unlike standard neural networks, smoothed classifiers can guarantee a correspondence from prediction confidence to adversarial robustness - which is the key motivation of our method in this paper. Figure 1 plots this relationship shown by Cohen et al. (2019) as well as by some prior works (Lecuyer et al., 2019; Li et al., 2019) which also attempt to lower-bound the robustness of smoothed classifiers.
sections.heading: 3 CONFIDENCE-AWARE TRAINING FOR RANDOMIZED SMOOTHING
sections.text: We aim to develop a new training method to maximize the certified robustness of f̂ , considering the trade-off relationship between robustness and accuracy (Zhang et al., 2019): even though randomized smoothing can be applied for any classifier f , the actual robustness of f̂ depends how much f classifies well under presence of Gaussian noise, i.e., by pf (x, y) as in (3). A simple way to train f for a robust f̂ , therefore, is to minimize the standard cross-entropy loss (denoted by CE below) with Gaussian augmentation as in Cohen et al. (2019):
  min F E (x,y)∼P δ∼N (0,σ2I) [CE(F (x+ δ), y)] . (5)
  In this paper, we extend this basic form of training to incorporate a confidence-aware strategy to decide which noise samples δ ∼ N (0, σ2I) should be focused on sample-wise during training of f . Recall Figure 1 that plots mappings from the (smoothed) confidence pf (4) to a certified radius, e.g., those derived by Cohen et al. (2019) (at the blue line). Ideally, one may wish to obtain a classifier f that achieves pf (x, y) ≈ 1 for every (x, y) ∼ P to maximize its certified robustness. In practice, however, such a case is highly unlikely, and there usually exists a sample x that pf (x, y) should be quite lower than 1 to maintain the discriminativity with other samples: in other words, these
  1Φ denotes the cumulative distribution function of the standard normal distribution.
  samples can be actually “beneficial” to be misclassified at some (hard) Gaussian noises, otherwise the classifier has to memorize the noises to correctly classify them. On the other hand, for the samples which can indeed achieve pf (x, y) ≈ 1, the current Gaussian training in (5) may not be able to provide enough samples of δ for x throughout the training, as pf (x, y) ≈ 1 implies that f(x+ δ) must be correctly classified for “almost every” possibility of δ ∼ N (0, σ2I). Also, considering that the radius certifiable at x rapidly increases as pf (x, y)→ 1 as shown in Figure 1, it is important for an overall robustness of f̂ to increase pf (x, y), especially when it can be close to 1 at the end.
  In these respects, we propose two different variants of Gaussian training (5) that address each of the possible cases, i.e., whether (a) pf (x, y) < 1 or (b) pf (x, y) ≈ 1, namely with (a) bottom-K and (b) worst-case Gaussian training, respectively. During training, the method first estimates pf (x, y) for each sample by simply computing their accuracy over M random samples of δ ∼ N (0, σ2I), and applies different forms of loss depending on the value. In the following two sections, i.e., Section 3.1 and 3.2, we provide the details on each of the proposed losses, and Section 3.3 describes how to combine the two losses and defines the overall training scheme.
  3.1 BOTTOM-K GAUSSIAN TRAINING: LOSS FOR LOW-CONFIDENCE SAMPLES
  Consider a base classifier f and a training sample (x, y) ∈ D, and suppose that pf (x, y) = p 1, e.g., f̂ has a low-confidence at x. Figure 2(a) visualizes this scenario: in this case, by definition of pf (x, y) in (4), f(x + δ) would be correctly classified to y only with probability p over δ, and this can imply either that (a) x + δ has not yet been adequately exposed to f during the training, or (b) x+ δ may be indeed hard to be correctly classified for some noise samples δ, so that minimizing the training loss at these noises could harm the classifier. The design goal of our proposed bottom-K Gaussian training is to modify the standard Gaussian training (5) to reduce the optimization burden from (b) while minimally retaining its ability to cover enough noise samples during training for (a).
  To this end, we first consider M random i.i.d. samples of δ, namely δ1, δ2, · · · , δM ∼ N (0, σ2I). Then, one can notice that the random variables 1[f(x + δi) = y]’s are also i.i.d., each of which follows the Bernoulli distribution of probability p, given that pf (x, y) = p. This means that, if the current pf (x, y) is the value one attempts to keep instead of further increasing it, the number of noise samples that should be correctly classified, which can be defined as ∑ i 1[f(x + δi) = y], would follow the binomial distribution, namely K ∼ Bin(M,p), and this motivates us to consider the following loss that only minimizes the K-smallest cross-entropy losses out of from M samples:
  Llow := 1
  M K∑ i=1 CE(F (x+ δπ(i)), y), where K ∼ Bin(M,pf (x, y)). (6)
  Here, π(i) denotes the noise index with the i-th smallest loss value in the M samples.
  Yet, the loss defined in (6) may not handle the cold-start problem on pf (x, y), e.g., at the early stage of the training where x + δ has not been adequately exposed to f , so that Llow can be minimized too early with an under-estimated p̂f . We found that, however, a simple trick of clamping K with 1 can bypass the issue: i.e., we always allow the “easiest” noise among the M samples to be fed into f throughout the training. In addition, we also found that it is still beneficial to constrain the samples that are not within K, i.e., the “harder” ones, to ensure that they still have a certain level of confidence for the class y even if they are misclassified: here, as a simple design, we propose to truncate the cross entropy loss with log p0 so that Fy(x+ δ) can be at least p0, e.g., p0 = 120 as done in our experiments. In these respects, we re-define the loss in (6) as in the followings:
  Llow := 1
  M K+∑ i=1 CE(F (x+ δπ(i)), y) + M∑ i=K++1 [ CE(F (x+ δπ(i)), y) + log p0 ]+ , (7) where K ∼ Bin(M,p), K+ := max(1,K), [·]+ := max(0, ·), and p0 ∈ (0, 1] is a hyperparameter. Note that the loss becomes the standard Gaussian training as p0 → 1 and returns to (6) as p0 → 0.
sections.heading: 3.2 WORST-CASE GAUSSIAN TRAINING: LOSS FOR HIGH-CONFIDENCE SAMPLES
sections.text: Next, we focus on the case when pf (x, y) ≈ 1, i.e., f̂ has a high confidence at x, as illustrated in Figure 2(b). In contrast to the previous scenario in Section 3.1 (and Figure 2(a)), now the major
  drawback of Gaussian training (5) does not come from the abundance of hard noises during training, but from the sparseness of such noises: considering that one can only present a limited number of noise samples to f throughout its training, naı̈vely minimizing (5) may not cover some “potentially hard” noise samples, and this would result in a significant harm in certified radius especially at the regime of pf (x, y) ≈ 1 as shown in Figure 1. The purpose of worst-case Gaussian training is to overcome this lack of samples via an adversarial search around each of the noise samples.
  Specifically, given that we have M samples of δ as in (7), namely δ1, δ2, · · · , δM ∼ N (0, σ2I), we propose to modify (5) to find and minimize the worst-case noise (a) around an `2-ball for each noise as well as (b) among the M samples, instead of minimizing the average-case loss:
  Lhigh := max i max ‖δ∗i−δi‖2≤ε CE(F (x+ δ∗i ), y). (8)
  We use the projected gradient descent (PGD) (Madry et al., 2018) to solve the inner maximization in (8): namely, we perform a T -step gradient ascent from each δi with step size 2 · ε/T while projecting the perturbations to be in the `2-ball of size ε. Here, although T and ε can be hyperparameters that affect the inner maximization, we simply fix them in our experiments by ε = 1.0 and T = 4. We also make sure that the likelihood of the optimized δ∗ as i.i.d. Gaussian still remains high by simply normalizing the mean and standard deviation of δ∗ to follow those of the original δ.
  Comparison to SmoothAdv. The idea of incorporating an adversarial search for the robustness of smoothed classifiers has been also considered in previous works (Salman et al., 2019; Jeong et al., 2021): e.g., Salman et al. (2019) have proposed SmoothAdv that applies adversarial training (Madry et al., 2018) to a “soft” approximation of f̂ given f and M noise samples:
  x∗ = arg max ||x′−x||2≤
  ( − log ( 1
  M ∑ i Fy(x ′ + δi)
  )) . (9)
  Our method is different from the previous approaches in which part of the inputs is adversarially optimized: i.e., we directly optimize the noise samples δi’s instead of x, with no need to assume a soft relaxation of f̂ . This is due to our unique motivation of finding the worst-case Gaussian noise, and our experimental results in Section 4 further support the effectiveness of this approach.
sections.heading: 3.3 OVERALL TRAINING SCHEME
sections.text: Given the two losses Llow and Lhigh defined in Section 3.1 and 3.2, respectively, we now define the full objective of our proposed Confidence-Aware Training for Randomized Smoothing (CATRS). Overall, in order to differentiate how to combine the two losses per sample basis, we use the smoothed confidence pf (x, y) (4) as the guiding proxy: specifically, we aim to apply the worst-case loss of Lhigh only for the samples where pf (x, y) is already high enough. In practice, however, one does not have a direct access to the value of pf (x, y) during training, and we estimate this with the M noise samples2 as done for Llow and Lhigh, i.e., by p̂f (x, y) := 1M ∑M i=1 1[f(x+δi) = y]. Here, we set a simple condition of “p̂f (x, y) = 1” to activate Lhigh, and the final loss becomes:
  LCAT-RS := Llow + λ · 1[p̂f (x, y) = 1] · Lhigh, (10)
  where 1[·] is the indicator random variable, and λ > 0 is a hyperparameter. The complete procedure of computing our proposed CAT-RS loss can be found in Appendix A.
sections.heading: 4 EXPERIMENTS
sections.text: In this section, we evaluate the effectiveness of our proposed training scheme compared to existing state-of-the-art training methods for smoothed classifiers, based on two well-established benchmarks on MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky, 2009) extensively in compliance to the standard protocol of the previous works (Cohen et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021). Overall, the experiments show that our method can consistently outperform the previous best efforts to improve the average certified radius by (a) maximizing the robust radii of high-confidence samples while (b) better maintaining the accuracy at low-confidence samples. We
  2In our experiments, we use M = 4 for our method unless otherwise noted.
  also perform a thorough ablation study on our loss design and confirm that (a) both of our proposed components have effectiveness to improve the certified robustness, and (b) the single hyperprameter λ (10) between the two losses can balance the trade-off between robustness and accuracy.
  Training setups. For a fair comparison, we follow the training setup considered in most of the previous works to compare the performance of the smoothed classifiers (Cohen et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021): specifically, we mainly consider LeNet (LeCun et al., 1998) and ResNet-110 (He et al., 2016) for MNIST and CIFAR-10, respectively, and consider three different scenarios of σ = 0.25, 0.5, and 1.0 for randomized smoothing. We train each model for 150 epochs with stochastic gradient descent (SGD) with a momentum of 0.9. The learning rates are initialized to 0.01 for MNIST and 0.1 for CIFAR-10, and decreased by a factor of 0.1 in every 50 epochs. We apply the same σ for both training and evaluation. More details on the training setups, e.g., hyperparameters, can be found in Appendix B.
  Baselines. We compare our method with an extensive list of baseline methods in the literature for training smoothed classifiers: (a) Gaussian training (Cohen et al., 2019) simply trains a classifier with Gaussian augmentation (5); (b) Stability training (Li et al., 2019) adds a cross-entropy term between the logits from clean and noisy images; SmoothAdv (Salman et al., 2019) employs adversarial training for smoothed classifiers (9); (d) MACER (Zhai et al., 2020) adds a regularization that aims to maximize a soft approximation of certified radius; (e) Consistency (Jeong & Shin, 2020) regularizes the variance of confidences over Gaussian noise; (f) SmoothMix (Jeong et al., 2021) proposes a mixup-based (Zhang et al., 2018) adversarial training for smoothed classifiers. Whenever possible, we use the pre-trained models publicly released by the authors to reproduce the results.
  Evaluation metrics. We follow the standard evaluation protocol for smoothed classifiers (Salman et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021): specifically, Cohen et al. (2019) have proposed a practical Monte Carlo based certification procedure, namely CERTIFY, that returns the prediction of f̂ and a lower bound of certified radius, CR(f, σ, x), over the randomness of n samples with probability at least 1 − α, or abstains the certification. Based on CERTIFY, we consider two major evaluation metrics: (a) the average certified radius (ACR) (Zhai et al., 2020): the average of certified radii on the test datasetDtest while assigning incorrect samples as 0, namely ACR := 1|Dtest| ∑ (x,y)∈Dtest [CR(f, σ, x) ·1f̂(x)=y], and (b) the approximate certified test accuracy at r: the fraction of the test dataset which CERTIFY classifies correctly with the radius larger than r without abstaining. We use n = 100, 000, n0 = 100, and α = 0.001 for CERTIFY, following the previous works (Cohen et al., 2019; Salman et al., 2019; Jeong & Shin, 2020; Jeong et al., 2021).
sections.heading: 4.1 RESULTS ON MNIST
sections.text: We compare the certified robustness of the smoothed classifiers on MNIST from our method to those from other baselines in Table 1. We also present in Figure 3 the plots of the approximate certified accuracy across varying r for σ ∈ {0.25, 0.5, 1.0}. Overall, the results show that our method of CAT-RS clearly surpasses all the other baselines in terms of ACR: i.e., our method could better balance between the clean accuracy and robustness. For σ = 0.25, we notice that some baselines, i.e., SmoothAdv and SmoothMix, already achieve a reasonably saturated level of ACR: even in this trivial task, our method could further improve the robust accuracy at r = 0.75 as 96.8% → 97.0%. In more challenging cases of σ = 0.5 and σ = 1.0, on the other hand, the improvements from CATRS in ACR become more evident as σ increases: e.g., at σ = 1.0, compared to SmoothMix (the bestperforming baseline), CAT-RS could improve the certified accuracy at r = 2.50 by 28.9%→ 29.8% while even improving the clean accuracy (i.e., certified accuracy at r = 0.0) by 93.7% → 93.9%. This means that our proposed CAT-RS can be more effective at challenging tasks, where it is more likely that a given classifier gets a more diverse confidence distribution for the training samples, so that our proposed confidence-aware training can better play its role.
  Accuracy-robustness trade-off. To further validate that our method can exhibit a better trade-off between accuracy and robustness compared to other methods, we additionally compare the performance trends between clean accuracy and certified accuracy at r = 2.0 as we vary a hyperparameter to control the trade-off, e.g., λ (10) in case of our method. We use σ = 1.0 for this experiment. We choose Consistency (Jeong & Shin, 2020) and SmoothMix (Jeong et al., 2021) for this comparison, considering that they also offer a single hyperparameter (namely λ and η, respectively) for the balance between accuracy and robustness similar to our method, while both generally achieve good performances among the baselines considered. The results plotted in Figure 5 clearly show that CAT-RS indeed exhibits a higher trade-off frontier compared to both methods, which confirms the effectiveness of our method. More detailed results can be found in Appendix C.
sections.heading: 4.2 RESULTS ON CIFAR-10
sections.text: Table 2 shows the performance of the baselines and our model on CIFAR-10. In Figure 4, we also plot the approximate certified accuracy over the range of r for σ ∈ {0.25, 0.5, 1.0}. In this experiment, for each σ, the baseline models are individually chosen based on its “best” ACR, i.e.,
  Consistency SmoothMix CAT-RS (ours)Ce rti fie
  d Ac
  c. @
  R =
  2 .0
  (% )
  35
  40
  45
  50
  Clean Acc. (%) 92 93 94 95 96 97
  Figure 5: Comparison of clean vs. certified accuracy at r = 2.0 on MNIST (σ = 1.0) for methods those offer trade-off control.
  1/p0 = 1 1/p0 = 2 1/p0 = 4 1/p0 = 8 1/p0 = 16 1/p0 = 32 1/p0 = ∞
  C er
  tif ie
  d ac
  cu ra
  cy
  0
  0.2
  0.4
  0.6
  Radius 0 0.5 1.0 1.5 2.0
  (a) Effect of p0
  λ= 0.25 λ= 0.50 λ= 1.00 λ= 2.00 λ= 4.00
  C er
  tif ie
  d ac
  cu ra
  cy
  0
  0.2
  0.4
  0.6
  Radius 0 0.5 1.0 1.5 2.0
  (b) Effect of λ
  Figure 6: Comparison of certified accuracy of CAT-RS ablations on CIFAR-10. We use ResNet-20 for ablation study and plot the results at σ = 0.5. More results can be found in Appendix D.
  the hyperparameter of the same baseline may vary over σ. For example, we choose the SmoothAdv baseline as the best model from the hundreds of hyperparameter configurations those examined by Salman et al. (2019) for each σ. Overall, our method of CAT-RS achieves a significant improvement of ACR compared to the baselines. In case of σ = 0.25 and σ = 0.5, for example, CAT-RS clearly offers a better trade-off between the clean accuracy and robustness compared to SmoothAdv, in a sense that (a) they achieve similar certified accuracy at large r, yet (b) CAT-RS could maintain much higher clean accuracy, e.g., 53.2% → 60.2% in case of σ = 0.5. For σ = 1.0, the ACR of our method significantly surpasses the previous best model, SmoothMix, by 0.773 → 0.812. As in MNIST, the improvement of CAT-RS is most evident in σ = 1.0, demonstrating the effectiveness of confidence-aware training. Furthermore, we observe that the performance gap between our method and the previous best model is more significant in CIFAR-10 because the higher complexity of CIFAR-10 compared to MNIST makes the confidence information more critical.
sections.heading: 4.3 ABLATION STUDY
sections.text: We also conduct an ablation study to further analyze individual effectiveness of the design components in our method. Unless otherwise specified, we use ResNet-20 (He et al., 2016) throughout this section and test it on a subsampled CIFAR-10 of size 500. We assume λ = 1.0 and p0 = 110 by default. We report the detailed results for this study in Appendix D.
  Effect of p0. We introduce a hyperparameter p0 in (7) to control how much we constrain the confidence of “hard” noises, so that p0 → 1 would lead the proposed Llow (7) into the standard
  Gaussian training. To verify its effectiveness, we conduct an experiment comparing the certified robustness of models with different p0 ∈ {0, 132 , 1 16 , 1 8 , 1 4 , 1 2 , 1} on σ ∈ {0.25, 0.5, 1.0}: Table 5 in Appendix D summarizes the results. Overall, we observe that setting p0 < 1 is clearly beneficial to improve ACR for all the σ considered, e.g., in case of σ = 1.0 the method achieve the best ACR when p0 = 116 , 1 32 , and this confirms that our proposed loss of L
  low (7) alone is superior to the Gaussian training even without the effect from Lhigh (8). From the Figure 6(a) which plots the results at σ = 0.5, one can observe that letting p0 → 0 leads the classifier to achieve a better robust accuracy with a relatively little degradation in the clean accuracy.
  Effect of λ. By the definition in (10), λ controls the contribution of Lhigh. We evaluate the impact of λ here, and the results are shown in Figure 6(b). We compare the performance of the models, varying λ ∈ {0.25, 0.5, 1.0, 2.0, 4.0} on σ = 0.5. As expected, we observe that λ balances the trade-off between robustness and clean accuracy; as λ increases, robustness increases while clean accuracy decreases. Also, in Table 7 in Appendix C, we verify that CAT-RS offers more effective trade-off between robustness and the clean accuracy than other methods. Further details can be found in Table 6 in Appendix D.
  Loss design. Our loss design in (10) combines several important ideas proposed in Section 3, and here we validate that each of the components has an individual effect in improving the certified robustness. In Table 3, we compare several variants of our proposed CAT-RS loss (10), namely (a) using only Llow (7), (b) using only Lhigh (8), and (c) Llow +λ ·Lhigh, i.e., not applying the masking condition 1[p̂f (x, y) = 1] to Lhigh. We also consider (d) a variant of Lhigh, where the outer max operation over noise samples (8) is replaced by the average, i.e., we define:
  Lavg max := 1
  M ∑ i ( max ‖δ∗i−δi‖2≤ε CE(F (x+ δ∗i ), y) ) . (11)
  We report the experimental results on σ = 0.5. Overall, CAT-RS achieves the best ACR and approximate certified accuracy among the variants. We observe that Llow alone improves Gaussian baseline, which highlights the importance of the confidence information for robust training. Also, Lhigh alone increases the robustness dramatically, while it sacrifices the clean accuracy. Combining Llow and Lhigh achieves better results by balancing robustness and accuracy. CAT-RS further improves the overall performance by simply masking out Lhigh for low-confidential predictions: i.e., when the confidence is not high enough, learning challenging adversarial samples harm the performance. By comparing the results made by using Lavg max and Lhigh, we verify that it is more helpful for a model to utilize the most challenging adversarial examples instead of averaging them.
sections.heading: 5 CONCLUSION
sections.text: This paper explores a close relationship between confidence and robustness, a natural property of smoothed classifiers yet neural networks cannot currently offer. We have successfully leveraged this relationship to relax the hard-to-compute metric of adversarial robustness into an easier concept of prediction confidence. Consequently, we could propose a practical robust training method that enables a sample-level control of adversarial robustness, which has been difficult without heuristics in a conventional belief. We believe our work could be a useful step for the future research on exploring the interesting connection between adversarial robustness and confidence calibration (Guo et al., 2017; Lee et al., 2018) through the framework of randomized smoothing.
  ETHICS STATEMENT
  Deploying deep learning based systems into the real-world, especially when they are of securityconcerned (Caruana et al., 2015; Yurtsever et al., 2020), still poses many potential risks for both companies and customers, and we researchers are responsible to make this technology more reliable through research towards AI safety (Amodei et al., 2016). Adversarial robustness is one of the central parts of this direction, and we believe our research on certified robustness can be a useful step towards building a practical yet secure deep learning system. Nevertheless, one should also recognize that adversarial robustness is still a bare minimum requirement for reliable deep learning, and the future research should address how to extend this restrictive notion of robustness into other challenging setups, e.g., corruption robustness (Hendrycks et al., 2020) and unrestricted attacks (Bhattad et al., 2020), just to name a few, to establish a practical sense of security for practitioners.
  REPRODUCIBILITY STATEMENT
  For the best practices to maintain the reproducibility of this paper, we have conducted our experiments on publicly available datasets, with the detailed descriptions for pre-processing in Appendix B.1. We have documented a thorough specification on the experimental details, e.g., training setups, baselines, and evaluation metrics, in Section 4. We have also fully specified all the hyperparameters considered to reproduce the baselines as well as our results in Appendix B.2. The code to reproduce our results will be publicly available, as well as the pre-trained models for our method.
sections.heading: A TRAINING PROCEDURE OF CAT-RS
sections.text: Algorithm 1 Confidence-Aware Training for Randomized Smoothing (CAT-RS) Require: training sample (x, y). smoothing factor σ. number of noise samples M > 0. truncation
  probability p0 ∈ (0, 1], regularization strength λ > 0. attack norm ε > 0.
  1: Sample δ1, · · · , δM ∼ N (0, σ2I) 2: p̂f ← 1M ∑ i 1[f(x+ δi) = y] 3: // COMPUTE THE BOTTOM-K LOSS 4: Sample K ∼ Bin(M, p̂f ) 5: K+ ← max(1,K) 6: for i = 1 to M do 7: Li ← CE(F (x+ δi), y) 8: end for 9: Lπ1:M ← argsort(L1:M )
  10: Llow ← 1M ( ∑K+ i=1 L π i + ∑M i=K++1(L π i + log p0)
  +) 11: // COMPUTE THE WORST-CASE LOSS 12: for i = 1 to M do 13: δ∗i ← arg max‖δ∗i−δi‖≤εCE(F (x+ δ ∗ i ), y) 14: end for 15: Lhigh ← maxiCE(F (x+ δ∗i ), y) 16: // COMPUTE THE CAT-RS LOSS 17: LCAT-RS ← Llow + λ · 1[p̂f = 1] · Lhigh
sections.heading: B EXPERIMENTAL DETAILS
sections.text: B.1 DATASETS
  MNIST (LeCun et al., 1998) consists of 70,000 gray-scale hand-written digit images of size 28×28, 60,000 for training and 10,000 for testing, where each is labeled to one value between 0 and 9. We do not perform any pre-processing except for normalizing the range of each pixel from 0-255 to 0-1. The dataset can be downloaded at http://yann.lecun.com/exdb/mnist/.
  CIFAR-10 (Krizhevsky, 2009) consists of 60,000 RGB images of size 32×32 pixels, 50,000 for training and 10,000 for testing, where each is labeled to one of 10 classes. We use the standard dataaugmentation scheme of random horizontal flip and random translation up to 4 pixels, following the practice of other baselines (Cohen et al., 2019; Salman et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021). We also normalize the images in pixel-wise by the mean and the standard deviation calculated from the training set. The full dataset can be downloaded at https: //www.cs.toronto.edu/˜kriz/cifar.html.
  B.2 HYPERPARAMETERS
  Stability training (Li et al., 2019) uses a single hyperparameter γ to control the relative strength of the regularization term. We fix γ = 2 for MNIST experiments. For CIFAR-10 experiments, γ = 2 is used for σ = 0.25, 0.5, and γ = 1 is used for σ = 1.0.
  SmoothAdv (Salman et al., 2019) uses three major hyperparameters to perform the projected gradient descent: namely, the attack radius in terms of `2-norm ε, the number of PGD steps T , and the number of noises m. In our experiments, we fix T = 10. For MNIST experiments, we fix ε = 1.0 and m = 4 as well. In case of CIFAR-10, on the other hand, we report the results chosen among the list of “best” configurations for each noise level which are previously searched by Salman et al. (2019): specifically, we report the results of ε = 1.0 and m = 4 for σ = 0.25, and ε = 2.0 and m = 2 for σ = 0.5, 1.0. When SmoothAdv is used, we adopt the warm-up strategy, i.e., we initially set ε = 0.0 and linearly increase to the target value of ε for 10-epochs.
  MACER (Zhai et al., 2020) introduces four hyperparameters: namely, the number of noises k, the coefficient for the regularization term λ, the clamping parameter for maximizing the certified radius γ, and the temperature scaling parameter β. For the MNIST experiments, we use k = 16, γ = 8.0, β = 16.0, and λ = 16.0 when σ = 0.25, 0.5, following the configurations in Zhai et al. (2020). When σ = 1.0, for the training to succeed, we set λ = 6.0. For the CIFAR-10 experiments, we follow the original configurations used by Zhai et al. (2020). We set k = 16, γ = 8.0, and β = 16.0. λ is set to be 12.0 and 4.0 for σ = 0.25 and 0.5, respectively. For σ = 1.0, the training starts with λ = 0 until the first learning rate decay and we set λ = 12.0 thereafter.
  Consistency (Jeong & Shin, 2020) uses two hyperparameters: namely, the coefficient for the consistency term η and those for the entropy term γ. We report the best results in terms of ACR among those reported by Jeong & Shin (2020) varying η. Following the original practice, we fix γ = 0.5 throughout our experiments. For MNIST, we use λ = 10 for σ = 0.25 and λ = 5 for other noises. For the CIFAR-10 experiments, we use λ = 20 for σ = 0.25 and λ = 10 for other noises.
  SmoothMix (Jeong et al., 2021) introduces four hyperparameters: namely, the coefficient for the mixup loss η, the step size for adversarial attack α, the number of steps for adversarial attack T , and the number of noises T . For the MNIST experiments, we fix η = 5.0, α = 1.0, and m = 4. T = 2, 4, 8 are used for the models with σ = 0.25, 0.5, 1.0, respectively. For the CIFAR-10 experiments, we again report the best result among those reported from Jeong et al. (2021): i.e., we fix η = 5.0,m = 2, and T = 4, and use α = 0.5, 1.0, 2.0 for σ = 0.25, 0.5, 1.0, respectively. The “one-step adversary” is used for σ = 0.5, 1.0 to follow the “best” configurations.
  CAT-RS (Ours). CAT-RS uses two main hyperparameters: namely, the coefficient for “worst-case” loss λ and the clamping parameter for maximizing the confidence of “hard” samples p0. For the MNIST experiments, we use the fixed configuration of λ = 1.0 and p0 = 15 . For the CIFAR-10 experiments, we use λ = 0.5, p0 = 120 for σ = 0.25, and λ = 2.0, p0 = 1 10 for σ = 0.5 and 1.0.
sections.heading: C COMPARISON OF ACCURACY-ROBUSTNESS TRADE-OFF
sections.heading: D DETAILED RESULTS ON ABLATION STUDY