summaries: The paper proposes a new federated learning algorithm called SDA-FL, which utilizes GANs to generate synthetic data for federated training on non-IID data. Specifically, each client pretrains a GAN to generate synthetic data and send them to the server. In each round, the server sends the global model and the synthetic data to the clients. Then, the clients update the label of the synthetic data and use both the local data and synthetic data to update the local model. The local models are sent to the server, which further averages the models and uses the averaged model to label the synthetic data. Experiments show that SDA-FL significantly outperforms the other federated learning approaches on non-IID data. The paper also studies the influence of different privacy budgets on the performance when applying differentially private GANs.

summaries: Federated learning suffers from non-IIDness across clients. A line of previous works address the non-IID problem by data sharing which violates the privacy requirement. In this paper, the author propose SDA-FL. Compare to the most basic FedAvg, the additional components includes  - *Image synthesis with DP-GAN*. Before training classifier, each client trains a differentially private GAN to generate synthetic data, and upload these data to PS for future data-sharing.  - *Synthetic image labeling*. Similar to self-training, PS use local models to assign pseudo-labels to unlabeled data. These labels are updated during training, i.e., interplay between model training and synthetic dataset updating.  - *Mixup*. For each clients, private data and shared synthetic data are mixed to alleviate non-IIDness.  - *ServerUpdate*. Data sharing also makes it possible for the server to conduct gradient descent.   This paper also empirically evaluate their framework under supervised and semi-supervised learning settings. Moreover, this paper study the sensitivity to privacy budget and effect of synthetic data.

summaries: This paper focused on a classical federated learning setting where data was non-iid partitioned in different local servers, and came up with a method named SDA-FL, which combined GAN-generated data, differential privacy to both address the non-iid problem and keep local data privacy. The main contributions are: 1. Utilized differential private GAN-generated data to solve the non-iid and local data privacy problems; 2. Designed a label updating mechanism to increase the model performance; 3. Tested the algorithm on CIFAR-10, MNIST, fashion-MNIST to confirm the performance of the algorithm.

summaries: This paper proposes a new framework for federated learning to resolve the non-IID issue by sharing differentially private synthetic data. Each client pretrains a local GAN to generate synthetic data and upload the data to the parameter server. To effectively use the synthetic data, the server performs pseudo labeling and shares this information with the clients along with the global model parameters, to let the local data, including both the real and synthetic data, approach an IID distribution. The interplay between model training and synthetic data updating improves the convergence of the local models and resolves the non-IID issue.

abstractText: Federated learning (FL) has recently emerged as a popular privacy-preserving collaborative learning paradigm. However, it suffers from the non-IID (independent and identically distributed) data among clients. In this paper, we propose a novel framework, namely Synthetic Data Aided Federated Learning (SDA-FL), to resolve the non-IID issue by sharing differentially private synthetic data. Specifically, each client pretrains a local generative adversarial network (GAN) to generate synthetic data, which are uploaded to the parameter server (PS) to construct a global shared synthetic dataset. The PS is responsible for generating and updating high-quality labels for the global dataset via pseudo labeling with a confident threshold before each global aggregation. A combination of the local private dataset and labeled synthetic dataset leads to nearly identical data distributions among clients, which improves the consistency among local models and benefits the global aggregation. To ensure privacy, the local GANs are trained with differential privacy by adding artificial noise to the local model gradients before being uploaded to the PS. Extensive experiments evidence that the proposed framework outperforms the baseline methods by a large margin in several benchmark datasets under both the supervised and semi-supervised settings.

authors:
Manoj Ghuhan Arivazhagan
Vinay Aggarwal
Aaditya Kumar Singh
Sunav Choudhary

title: Federated learning with personalization layers

venue: arXiv preprint arXiv:1912.00818,

authors:
Martin Arjovsky
Léon Bottou

title: Towards principled methods for training generative adversarial networks

venue: arXiv preprint arXiv:1701.04862,

authors:
Christopher Bowles
Liang Chen
Ricardo Guerrero
Paul Bentley
Roger Gunn
Alexander Hammers
David Alexander Dickie
Maria Valdés Hernández
Joanna Wardlaw
Daniel Rueckert. Gan

title: augmentation: Augmenting training data using generative adversarial networks

venue: arXiv preprint arXiv:1810.10863,

authors:
Dingfan Chen
Ning Yu
Yang Zhang
Mario Fritz

title: Gan-leaks: A taxonomy of membership inference attacks against generative models

venue: In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security,

authors:
Muhammad EH Chowdhury
Tawsifur Rahman
Amith Khandakar
Rashid Mazhar
Muhammad Abdul Kadir
Zaid Bin Mahbub
Khandakar Reajul Islam
Muhammad Salman Khan
Atif Iqbal
Nasser Al Emadi

title: Can ai help in screening viral and covid-19 pneumonia

authors:
Charles Corbière
Nicolas Thome
Avner Bar-Hen
Matthieu Cord
Patrick Pérez

title: Addressing failure prediction by learning model confidence

venue: In Conference on Neural Information Processing Systems (NIPS),

authors:
Yuyang Deng
Mohammad Mahdi Kamani
Mehrdad Mahdavi

title: Adaptive personalized federated learning

venue: arXiv preprint arXiv:2003.13461,

authors:
Enmao Diao
Jie Ding
Vahid Tarokh

title: Semifl: Communication efficient semi-supervised federated learning with unlabeled clients

venue: arXiv e-prints, pp

authors:
Moming Duan
Duo Liu
Xianzhang Chen
Yujuan Tan
Jinting Ren
Lei Qiao
Liang Liang

title: Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications

venue: IEEE 37th international conference on computer design (ICCD),

title: Differential privacy: A survey of results

venue: In International conference on theory and applications of models of computation,

authors:
Z. Eaton-Rosen
Felix J.S. Bragman
Sébastien Ourselin
M. Cardoso

title: Improving data augmentation for medical image segmentation

venue: Conference on Medical Imaging with Deep Learning (MIDL),

authors:
Avishek Ghosh
Jichan Chung
Dong Yin
Kannan Ramchandran

title: An efficient framework for clustered federated learning

authors:
Ian Goodfellow
Jean Pouget-Abadie
Mehdi Mirza
Bing Xu
David Warde-Farley
Sherjil Ozair
Aaron Courville
Yoshua Bengio

title: Generative adversarial networks

venue: Communications of the ACM,

authors:
Ishaan Gulrajani
Faruk Ahmed
Martı́n Arjovsky
Vincent Dumoulin
Aaron C Courville

title: Improved training of wasserstein gans

venue: In NIPS,

authors:
Kaiming He
Xiangyu Zhang
Shaoqing Ren
Jian Sun

title: Deep residual learning for image recognition

venue: Conference on Conference on Computer Vision and Pattern Recognition

authors:
Martin Heusel
Hubert Ramsauer
Thomas Unterthiner
Bernhard Nessler
Günter Klambauer
Sepp Hochreiter

title: Gans trained by a two time-scale update rule converge to a nash equilibrium

authors:
Sai Praneeth Karimireddy
Satyen Kale
Mehryar Mohri
Sashank J Reddi
Sebastian U Stich
Ananda Theertha Suresh

title: Scaffold: Stochastic controlled averaging for federated learning

venue: In ICML,

authors:
Kavya Kopparapu
Eric Lin

title: Fedfmc: Sequential efficient federated learning on non-iid data

venue: arXiv preprint arXiv:2006.10937,

title: Learning multiple layers of features from tiny images

venue: Master’s thesis,

authors:
Yann LeCun
Léon Bottou
Yoshua Bengio
Patrick Haffner

title: Gradient-based learning applied to document recognition

venue: Proceedings of the IEEE,

authors:
Tian Li
Anit Kumar Sahu
Manzil Zaheer
Maziar Sanjabi
Ameet Talwalkar
Virginia Smith

title: Federated optimization in heterogeneous networks

venue: Proceedings of Machine Learning and Systems (MLSys),

authors:
Xiang Li
Kaixuan Huang
Wenhao Yang
Shusen Wang
Zhihua Zhang

title: On the convergence of fedavg on non-iid data

venue: In International Conference on Learning Representations,

authors:
Paul Pu Liang
Terrance Liu
Liu Ziyin
Nicholas B Allen
Randy P Auerbach
David Brent
Ruslan Salakhutdinov
Louis-Philippe Morency

title: Think locally, act globally: Federated learning with local and global representations

venue: arXiv preprint arXiv:2001.01523,

authors:
Tao Lin
Lingjing Kong
Sebastian U Stich
Martin Jaggi

title: Ensemble distillation for robust model fusion in federated learning

authors:
H. Brendan McMahan
Eider Moore
Daniel Ramage
Seth Hampson
Blaise Agüera y Arcas

title: Communication-efficient learning of deep networks from decentralized data

venue: International Conference on Artificial Intelligence and Statistics (AISTATS),

authors:
Mehdi Mirza
Simon Osindero

title: Conditional generative adversarial nets

venue: arXiv preprint arXiv:1411.1784,

authors:
Yuval Netzer
Tao Wang
Adam Coates
Alessandro Bissacco
Bo Wu
Andrew Y Ng

title: Reading digits in natural images with unsupervised feature learning

authors:
Dinh C Nguyen
Ming Ding
Pubudu N Pathirana
Aruna Seneviratne
Albert Y Zomaya

title: Federated learning for covid-19 detection with generative adversarial networks in edge cloud computing

venue: IEEE Internet of Things Journal,

authors:
Augustus Odena
Christopher Olah
Jonathon Shlens

title: Conditional image synthesis with auxiliary classifier gans

venue: In International conference on machine learning,

authors:
Seungeun Oh
Jihong Park
Eunjeong Jeong
Hyesung Kim
Mehdi Bennis
Seong-Lyun Kim

title: Mix2fld: Downlink federated learning after uplink federated distillation with two-way mixup

venue: IEEE Communication Letters,

authors:
Sinno Jialin Pan
Qiang Yang

title: A survey on transfer learning

venue: IEEE Transactions on knowledge and data engineering,

authors:
Alec Radford
Luke Metz
Soumith Chintala

title: Unsupervised representation learning with deep convolutional generative adversarial networks

venue: arXiv preprint arXiv:1511.06434,

authors:
Daniel Rothchild
Ashwinee Panda
Enayat Ullah
Nikita Ivkin
Ion Stoica
Vladimir Braverman
Joseph Gonzalez
Raman Arora

title: Fetchsgd: Communication-efficient federated learning with sketching

venue: In International Conference on Machine Learning,

authors:
MyungJae Shin
Chihoon Hwang
Joongheon Kim
Jihong Park
Mehdi Bennis
Seong-Lyun Kim

title: Xor mixup: Privacy-preserving data augmentation for one-shot federated learning. International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2020 (FL-ICML’20), 2020

authors:
Kihyuk Sohn
David Berthelot
Nicholas Carlini
Zizhao Zhang
Han Zhang
Colin A Raffel
Ekin Dogus Cubuk
Alexey Kurakin
Chun-Liang Li

title: Fixmatch: Simplifying semi-supervised learning with consistency and confidence

authors:
Reihaneh Torkzadehmahani
Peter Kairouz
Benedict Paten

title: Dp-cgan: Differentially private synthetic data and label generation

venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,

authors:
Hao Wang
Zakhary Kaplan
Di Niu
Baochun Li

title: Optimizing federated learning on non-iid data with reinforcement learning

venue: In IEEE INFOCOM 2020-IEEE Conference on Computer Communications,

authors:
Jianyu Wang
Qinghua Liu
Hao Liang
Gauri Joshi
H Vincent Poor

title: Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 2020b

authors:
Han Xiao
Kashif Rasul
Roland Vollgraf

title: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

venue: arXiv preprint arXiv:1708.07747,

authors:
Liyang Xie
Kaixiang Lin
Shu Wang
Fei Wang
Jiayu Zhou

title: Differentially private generative adversarial network

venue: arXiv preprint arXiv:1802.06739,

authors:
Tehrim Yoon
Sumin Shin
Sung Ju Hwang
Eunho Yang

title: Fedmix: Approximation of mixup under mean augmented federated learning

venue: arXiv preprint arXiv:2107.00233,

authors:
Naoya Yoshida
Takayuki Nishio
Masahiro Morikura
Koji Yamamoto
Ryo Yonetani

title: Hybridfl for wireless networks: Cooperative learning mechanism using non-iid data

venue: IEEE International Conference on Communications (ICC),

authors:
Felix Yu
Ankit Singh Rawat
Aditya Menon
Sanjiv Kumar

title: Federated learning with only positive labels

authors:
Hongyi Zhang
Moustapha Cisse
Yann N. Dauphin
David Lopez-Paz

title: mixup: Beyond empirical risk minimization

venue: International Conference on Learning Representations (ICLR),

authors:
Longling Zhang
Bochen Shen
Ahmed Barnawi
Shan Xi
Neeraj Kumar
Yi Wu

title: Feddpgan: Federated differentially private generative adversarial networks framework for the detection of covid-19 pneumonia

venue: Information Systems Frontiers,

authors:
Yue Zhao
Meng Li
Liangzhen Lai
Naveen Suda
Damon Civin
Vikas Chandra

title: Federated learning with non-iid data

venue: arXiv preprint arXiv:1806.00582,

authors:
Hangyu Zhu
Jinjin Xu
Shiqing Liu
Yaochu Jin

title: Federated learning on non-iid data: A survey

venue: arXiv preprint arXiv:2106.06843,

heading: 1 INTRODUCTION

text:
The recent development of deep learning technologies has led to major breakthroughs in various domains. This results in a tremendous amount of valuable data that can facilitate the training of deep learning models for intelligent applications. A traditional approach to exploit these distributed data samples is to upload them to a centralized server for model training. However, directly offloading data raises severe privacy concerns as data collected by mobile clients may contain sensitive information.
By decoupling model training from the need of transferring private data to the cloud, federated learning (FL) offers a promising approach to collaboratively learn a global model without directly sharing the local data. Particularly, McMahan et al. (2017) introduced the Federated Averaging (FedAvg) algorithm that synchronously aggregates the updates of local models, while satisfying the basic requirements for privacy protection. Later, FedProx was proposed by Li et al. (2020) that adds a proximal term in the loss function to improve the statistical stability of the training process.
A key characteristic of federated learning is the non-IID (independent and identically distributed) data distribution among clients. Consequently, different clients learn from different data distributions, which leads to high heterogeneity among local models and degrades the effectiveness of model aggregation (McMahan et al., 2017; Wang et al., 2020b). For extreme cases with highly skewed data distribution, traditional federated learning methods even lack a theoretical guarantee of convergence (Yu et al., 2020; Rothchild et al., 2020). Recent studies have attempted to address the non-IID issue by constructing a global shared dataset based on the local data to mitigate the data imbalance effect (Krizhevsky et al., 2009; Zhao et al., 2018; Yoshida et al., 2020; Oh et al., 2020). Particularly, several methods (Krizhevsky et al., 2009; Yoshida et al., 2020) proposed data sharing strategies to distribute a percentage of raw data among clients, but naive data sharing seriously violate the privacy requirement in FL. Besides, data augmentation approaches (Zhang et al., 2021; Oh et al., 2020) generate synthetic samples without directly sharing private data. However, these methods are still susceptible to data leakage without a privacy-preserving mechanism. Thus, it is important to investigate how
to generate privacy-preserving synthetic samples, as well as how to effectively utilize the private dataset and synthetic dataset in the FL framework.
To alleviate the heterogeneity of the data distribution in FL without compromising privacy, we propose a novel framework, named Synthetic Data Aided Federated Learning (SDA-FL), which resolves the non-IID issue by sharing differentially private synthetic data. Under this framework, each client pretrains a local generative adversarial network (GAN) (Goodfellow et al., 2020) to generate synthetic data, thus avoiding sharing the raw data directly. A parameter server (PS) performs pseudo-labeling for the uploaded synthetic data and constructs a global shared dataset. In each global aggregation, the PS updates the pseudo labels of the synthetic dataset based on the received models and updates the global model with the confident synthetic data. This iterative labeling process constitutes a synthetic dataset with high-confidence labels, which is fed back to the clients to construct their training samples, including the local private data and synthetic data, following an almost identical distribution. Meanwhile, to provide a privacy guarantee for the generative models, we adopt the widely used differential privacy (Dwork, 2008) to ensure that the actual training samples can not be revealed by adversaries. In particular, we use the Wasserstein GAN with gradient penalty (WGAN-GP) (Gulrajani et al., 2017) as the backbone and introduce the privacy budget by adding artificial noise to the gradients in the training process. The SDA-FL framework is mostly compatible with many existing FL methods and can be applied in both supervised and semi-supervised settings.
We evaluate the proposed framework on several standard datasets under both supervised learning and semi-supervised learning, which shows its effectiveness in resolving the non-IID distribution problem even in the extreme case (with only one class of real data at each client). We also investigate the tradeoff between the privacy budget and the FL performance, which demonstrates that this framework is fully functional under the strict differential privacy protection requirement. Our major contributions are summarized as follows:
• We develop a global shared synthetic dataset by the differentially private GAN models. This data augmentation method effectively resolves the non-IID problem in FL and maintains the local data privacy.
• To utilize the global shared synthetic dataset efficiently, we propose a novel framework, SDA-FL, in which the PS provides and keeps updating high-confidence pseudo labels for the synthetic dataset iteratively, thus improving the performance of the global model.
• We evaluate our framework on several benchmark datasets under the high heterogeneous data distribution, where our framework consistently outperforms other baselines under both supervised and semi-supervised learning. Besides, our framework experiences a slight performance degradation with a stringent privacy budget.

heading: 2 RELATED WORK

text:
Non-IID Challenges in Federated Learning The pioneering work in FL is FedAvg (McMahan et al., 2017), which simply averages model parameters trained by different clients in an elementwise manner, weighted proportionately by the clients’ data size. However, this algorithm often leads to slow convergence and performance degradation when data is heterogeneous across local clients (Li et al., 2019). To alleviate this issue, Wang et al. (2020a) leveraged the reinforcement learning to intelligently choose the client devices to participate in each round of federated learning to counterbalance the bias introduced by non-IID data. Besides, some studies proposed personalized layers to adjust the local model according to the private dataset (Arivazhagan et al., 2019; Liang et al., 2020). For example, FedPer was introduced in Arivazhagan et al. (2019) that captures personalization aspects in federated learning by only using the local data to train the personalized layers. Knowledge distillation (Deng et al., 2020; Lin et al., 2020) is also a promising idea of personalized federated learning, which transfers information from other clients to a certain client. Although the aforementioned methods outperform FedAvg on strongly non-IID data, the extra computational costs are not affordable by the resource-constrained IoT devices (Zhu et al., 2021). Thus, client clustering has been proposed to construct a multi-center framework by grouping the clients into different clusters based on the model similarity (Ghosh et al., 2020; Kopparapu & Lin, 2020). As clients avoid the negative knowledge transfer from the dissimilar models (Pan & Yang, 2009), clustering methods can alleviate the negative effect of the non-IID data distribution. However, such methods are not scalable as the communication cost increases largely with the number of clients.
Data Augmentation and Privacy Preserving Recently, methods based on some form of data sharing have received increasing attentions and achieved prominent performance in FL (Zhu et al., 2021; Zhao et al., 2018). Particularly, data augmentation approaches showed the potential capability to address the non-IID issue by modifying the local distributions without sharing private data (Duan et al., 2019; Zhang et al., 2018; Eaton-Rosen et al., 2020). A vanilla data augmentation method was proposed by Duan et al. (2019), in which each client performs local data augmentation to reach a balanced global distribution. Besides, several methods (Oh et al., 2020; Shin et al., 2020; Yoon et al., 2021) leveraged the Mixup technique (Zhang et al., 2018) to conduct data augmentation, where the clients share the blended local data and collaboratively construct a new global dataset to tackle the non-IID issue. However, frequent data exchange may be susceptible to privacy issues. Recently, the GAN-based data augmentation framework (Bowles et al., 2018; Yoshida et al., 2020) was shown to be promising for reducing the degree of local data imbalance in FL. The general idea is to train a good generative model in the presence of non-IID data. Under this framework, each client sends its local seed samples to the server for the GAN training, and then a well-trained generator will be sent to all the clients for model updating. However, directly sending local data samples to the server violates the data privacy requirement. FedDPGAN (Zhang et al., 2021) suggested an alternative that all the clients train a global GAN together based on the FL framework to supplement the scarce local Covid-19 data. Unfortunately, the white-box generators sent to the server are vulnerable to adversarial attacks (Chen et al., 2020).

heading: 3 PRELIMINARY

text:
Federated Learning FedAvg (McMahan et al., 2017) is one of the representative training algorithms in federated learning, and it trains a shared neural network without disclosing the raw data of clients. For every step t = 0, . . . , T − 1, every client k ∈ St downloads the global model wt and updates the local model with local dataset Dk via stochastic gradient descent (SGD), i.e., wt+1
k ← wtk − ηt+1∇`. The updated local models are then sent back to the PS for a weighted aggregation wt+1 ← 1K ∑ k∈St w k t+1. These procedures repeat until convergence or all the T training steps are exhausted.
Differentially Private Generative Adversarial network We utilize a differentially private generative adversarial network (DPGAN) for data augmentation while protecting local data privacy. To avoid the gradients vanishing and mode collapse problems (Arjovsky & Bottou, 2017; Arjovsky et al., 2017) that are commonly encountered by many GAN models (Mirza & Osindero, 2014; Radford et al., 2015; Odena et al., 2017), we adopt the WGAN-GP (Gulrajani et al., 2017), which is upgraded based on the Wasserstein GAN (WGAN) (Arjovsky & Bottou, 2017; Arjovsky et al., 2017), as the backbone of the generative model for a stable training procedure. Besides, we add noise to the gradient in the generator training process, which provides privacy control in terms of differential privacy (Torkzadehmahani et al., 2019). The definition of differential privacy is as follows: Definition 1. (Differential privacy1): A random mechanism Ap satisfies ( ,δ)-differential privacy if for any output’s subset (S) and for any two adjacent datasets D, D′, the following probability inequality holds:
P (Ap(D) ∈ S) ≤ e · P (Ap (D′) ∈ S) + δ. (1)
Particularly, δ is an optimal probability that guarantees the validity of the inequality, and is the privacy budget indicating the privacy level, i.e., a smaller value of implies stronger privacy protection. To satisfy the ( , δ)-differential privacy, we follow the work (Xie et al., 2018) and add the Gaussian noise to the gradients updated in each discriminator training iteration. The relationship between the noise variance and differential privacy is shown as follows:
σn = 2q √ nd log ( 1
δ
) / , (2)
where q and nd denote the sample probability and the number of discriminator training iterations in each training round, respectively.
1Although our work leverages the differential privacy to address the extra privacy problem introduced by the GAN-based data augmentation, the potential privacy risk of FL is still an open problem, which is out of the scope of our study.
Pseudo Labeling Although GAN-based data augmentation can generate synthetic data, the labels of the synthetic data are unknown, which hinders the training of the model through backpropagation. In semi-supervised learning, pseudo labeling is a widely used method in classification tasks to generate confident pseudo labels (Sohn et al., 2020). A hard pseudo label will be applied to the unlabeled data if the maximum class probability (MCP) of the inference result (Corbière et al., 2019) exceeds a predefined threshold τ . In our experiments, the threshold is set to be 0.95.

heading: 4 SYNTHETIC DATA AIDED FEDERATED LEARNING (SDA-FL)

text:
We now introduce the SDA-FL framework that adopts GAN-based data augmentation to alleviate the negative effect of the non-IID data. The system diagram is shown in Fig. 1, and key algorithmic innovations built upon the classic FL framework are elaborated below.
Global Synthetic Dataset Construction At the beginning of the federated training, each client pretrains a local GAN model to generate synthetic samples based on its private data. Then, the synthetic samples are sent to the PS to construct a global shared synthetic dataset. To effectively leverage the synthetic dataset for federated learning, we perform pseudo labeling for these samples, which is critical for the effectiveness of the proposed framework.
In federated learning, the local model is the most suitable network to predict the pseudo labels for the synthetic data generated by the corresponding local GAN, as they are both trained with the same data. In our framework, to improve the confidence level of the pseudo labels, we utilize the local models to perform the pseudo labeling for the unlabeled synthetic data. Specifically, after receiving the local model wit+1 in each aggregation step, the PS utilizes the model predictions to assign a pseudo label for each unlabeled synthetic instance if the maximum class probability is higher than a predefined threshold τ . In this way, we are gradually generating synthetic data samples with high-quality labels.
Synthetic Data Aided Model Training Augmented by the samples from the shared synthetic dataset that are with the confident labels, the data available for local training at different clients reach approximately homogeneous distribution. To make good use of the synthetic data, we leverage the Mixup method (Zhang et al., 2018), which utilizes linear interpolation between the real sample (xi, yi) and the synthetic sample (x̂i, ŷi) to augment the real data:
x̄ = λ1x̂i + (1− λ1)xi, ȳ = λ1ŷi + (1− λ1)yi,
(3)
where λ1 follows the Beta distribution ( Beta(α, α) ) with α ∈ [0, 1]. Combined with the crossentropy loss, the Mixup loss for local update becomes:
`1 = λ1` ( f(X̄;wi), Ŷi ) + (1− λ1)` ( f(X̄;wi),Yi ) . (4)
Algorithm 1: Synthetic Data Augmented Federated Learning (SDA-FL) Input: Local dataset Dk = {Xk,Yk},
Pretrained generators G = {Gk} k ∈ {1, . . . , N}
Initialize w0,Synthetic dataset D̂s = {X̂s, Ŷs} for server, D̂k = {X̂k, Ŷk} for clients All clients upload the synthetic data generated by Gk to PS for t = 0, . . . , T − 1 do St ← K clients are selected at random Send wt and updated pseudo labels to
clients k ∈ St Clients Side: for k ∈ St do
Updata local synthesis dataset(X̂k, Ŷk) wkt+1 ← LocalUpdate(k,wt;Xk,Yk, X̂k, Ŷk)
end Parameter Server Side: Pseudo Label Update wt+1 ← 1K ∑ k∈St w k t+1
wt+1 ← ServerUpdate(wt+1; X̂s, Ŷs) end
Algorithm 2: Model Update
LocalUpdate(k,wt;Xk,Yk, X̂k, Ŷk) : w ← wt for e = 0, . . . , E − 1 do
Split Dk into batches of size B for batch(X,Y ) do
Sample batch(X̂, Ŷ ) from (X̂k, Ŷk)
Use (X̂, Ŷ ) and (X,Y ) to update w following (6)
end end return w ServerUpdate(wt+1; X̂s, Ŷs): w ← wt+1 for e = 0, . . . , E − 1 do
Sample batch(X̂, Ŷ ) from (X̂s, Ŷs) Evenly divide (X̂, Ŷ ) into (X̂1, Ŷ1)
and (X̂2, Ŷ2) Use (X̂1, Ŷ1) and (X̂2, Ŷ2) to update w following (6)
end return w
In addition, to keep more information of the local clients, another cross-entropy loss term is introduced for the real data as follows:
`2 = ` ( f(Xi;wi),Yi ) . (5)
Then SGD is applied to update the local model:
wi ← wi − ηt+1∇(`1 + λ2`2), (6) where λ2 is a hyperparameter to control the retention of the local information and will be set in different values in experiments. More details about the local update are shown in Algorithm 2.
In contrast to the traditional federated learning where the PS does not have access to any data to update the global model, the PS in our framework can keep the entire global synthetic dataset D̂s and uses it to update the global model, thereby helping to enhance the performance.
Interplay between Model Training and Synthetic Dataset Updating Because of the non-IID problem, there is a distinct difference in the weights of the local models among clients in federated learning. Furthermore, simply sharing the synthetic data with the PS and other clients will not result in the good training performance of the global model due to the lack of confident pseudo labels. Our framework, SDA-FL, deeply integrates federated learning and synthetic data, which effectively alleviates the non-IID issue and helps to train a good global model. As shown in Fig. 1, after creating the global synthetic dataset, it will be shared with all the clients so that the local data, including both the real and the synthetic data, approach an IID distribution. Without a doubt, the supplement of synthetic data results in the improvement of the local models. After being uploaded to the PS, the updated local models are used to perform the pseudo labeling and update the global synthetic dataset by the PS. The stronger local models can boost the confidence of the pseudo label. With the improvement of labeling for the global synthetic dataset, the PS can utilize it to update the global model and all the clients can improve their local models subsequently at the next-round training. Thus there is an interplay between the model update and the synthetic dataset at every training round: global synthetic dataset → global model update → local model update → labeling for global synthetic dataset, which results in a converged and well-perform model.
SDA-FL vs. Traditional FL From the above descriptions, the proposed SDA-FL framework introduces two additional operations at the server side, namely, pseudo label update of the shared synthetic dataset and server-side model update, as shown in Fig. 1. These two main innovations contribute to the performance improvement with non-IID data. In traditional FL algorithms (Li et al., 2020; Karimireddy et al., 2020), clients update their models based on local private data, which may lead to performance degradation when data are heterogeneous over clients. In our framework, the local dataset is augmented by the GAN-based synthetic samples, which alleviates the non-IID problem. Furthermore, the PS in the traditional FL system only performs model aggregation based on the uploaded local models. In contrast, our framework generates high-confidence synthetic data to update and improve the global model at the PS. Overall, with a shared synthetic dataset and an effective mechanism for pseudo label update, SDA-FL can overcome the heterogeneous data distributions among clients and enhance the model update at the server. We envision that this framework can be applied to develop other data augmentation-based methods for more efficient federated learning.

heading: 5 EXPERIMENTS

text: In this section, we evaluate the proposed SDA-FL framework in the presence of non-IID data for both supervised learning and semi-supervised learning. The experimental results on different benchmark datasets demonstrate the effectiveness of the proposed framework compared with baseline methods. Besides, ablation studies are conducted to illustrate the impacts of key factors on the generated samples and the model convergence.

heading: 5.1 EXPERIMENTAL SETUP

text:
Datasets We implement the SDA-FL framework in the scenario where all the real data are stored in clients and the PS only has access to the synthetic data. We use four benchmark datasets, including MNIST (LeCun et al., 1998), FashionMNIST (Xiao et al., 2017), Cifar-10 (Krizhevsky, 2009), and SVHN (Netzer et al., 2011) to test the proposed method. The details about the datasets, models, and experimental settings are deferred to Appendix A. To demonstrate the effectiveness of our framework in handling different non-IID scenarios, we provide extensive experiments in the following five tasks.
Tasks 1) Federated supervised learning: We compare our framework with FedAvg (McMahan et al., 2017), Fedprox (Li et al., 2020), Scaffold (Karimireddy et al., 2020), Naivemix, and Fedmix (Yoon et al., 2021) on MNIST, FashionMNIST, Cifar-10, and SVHN datasets. 2) Federated semi-supervised learning: We extend our framework to the semi-supervised learning setting in MNIST, FashionMNIST, and Cifar-10 datasets by performing pseudo labeling for the unlabeled data. To show the effectiveness of our SDA-FL method, we compare it with Local Fixmatch+Mixup (Diao et al., 2021), Local Fixmatch (Sohn et al., 2020), and Local Mixup (Zhang et al., 2018). 3) The trade-off between the privacy budget and performance: To investigate the influence of the differential privacy mechanism on the quality of the synthetic data and the model convergence, we validate our framework under different privacy budgets. In particular, we select the Fréchet Inception Distance (FID) (Heusel et al., 2017) as a metric to quantify the quality (similarity) of the synthetic data. 4) The effectiveness of updating pseudo labels and the server update: Compared with the traditional federated learning, SDA-FL leverages the pseudo labeling mechanism to generate highconfidence pseudo labels for the synthetic data, which are also used by the PS to further update the global model. To demonstrate the efficacy of these two mechanisms, we conduct ablation studies on a varying number of rounds for updating the pseudo labels and a varying number of server update steps. 5) Ablation studies of the computational cost and the number of samples for training the generators: To evaluate the performance of the generative models, we conduct ablation studies to investigate the influence of the training epochs and the number of samples on the quality (similarity) of the synthetic data.

heading: 5.2 PERFORMANCE OF SDA-FL ON NON-IID FEDERATED LEARNING

text:
1) Federated Supervised Learning We compare the performance of various methods under the same federated setting. With varying numbers of classes per client, the experimental results in Table 1 show that our framework outperforms other methods by a large margin, which is attributed to the GAN-based data augmentation that mitigates the detrimental effects of the data heterogeneity. Under the severe non-IID scenario (i.e., each client only has one class of data), our SDA-FL method maintains an accuracy of 88.46% in the SVHN classification task, while other baselines suffer from great performance degradation. In the Cifar-10 experiments, our framework outperforms the classic Naivemix and Fedmix algorithms at least by 5.0% with three classes of data at each client. This indicates that augmentation via synthetic data is more effective than averaging real data samples, although Fedmix adds another derivative term when averaging real images to explicitly handle the non-IID problem.
Recently, GANs have been widely used to solve the non-IID and data scarce issues based on the federated learning in the medical field (Nguyen et al., 2021; Zhang et al., 2021), which can afford the additional computational cost for training the GANs. To further show the feasibility of SDA-FL, we include another comparison on a Covid-19 dataset (Chowdhury et al., 2020). See more details about the Covid-19 experiment in Appendix B.
2) Federated Semi-Supervised Learning The results in Fig. 2 show that our framework achieves faster convergence and better performance than other algorithms, indicating its robustness and generalizability. Particularly, compared with Local Fixmatch+Local Mixup, our method improves the accuracy by nearly 10% in the FashionMNIST classification task. In the Cifar-10 dataset, the baseline methods are not able to train a usable global model (with a test accuracy below 40%), while our framework converges in this challenging scenario and improves the test accuracy by a significant margin. This is because the proposed pseudo labeling mechanism can provide high-quality labels for the synthetic and unlabeled samples, which benefits the federated training process.
3) The trade-off between the privacy budget and performance The privacy budget provides the privacy protection for the generation of the synthetic dataset, but a small value of affects the quality of the synthetic samples and thus degrades the training performance. To investigate the impact of the privacy budget, we evaluate the performance of the proposed method under different
values of . As shown in Fig.3, the proposed framework suffers from performance degradation with a small privacy budget. Compared with a protection-free scenario ( = ∞), setting a strict privacy budget ( = 5) (Xie et al., 2018) leads to around 0.61% and 2.59% accuracy drop in the MNIST and FashionMNIST datasets, respectively. Besides, we utilize the FID to measure the quality of the generated samples, where a smaller FID score denotes a better image quality. As illustrated in Fig.3, decreasing the privacy budget increases the FID score, which implies the degradation of the quality of the generated samples. Note that although our SDA-FL method is trained under the strict privacy requirement, compared with the results in Table 1, our framework still maintains desired performance over other methods.
In the Cifar-10 classification task, as the generators are sensitive to the Gaussian noise added to the gradient, we adopt a large privacy budget ( = 50) when training the proposed method. Fig.3(c) shows that our framework outperforms Fedmix and Fedprox in federated supervised learning. To illustrate the privacy guarantee on the generators in our framework, we show some synthetic samples in Fig.4(a), which are extracted from the generators trained with = 200. In comparison with the samples from the private datasets, the synthetic images are blurry and noisy, which means the GAN models are not memorizing the real data, and the local privacy is not undermined. Besides, the performance improvements shown in Fig.3(c) and Table 1 indicate that the synthetic images with low perceptual quality can capture the necessary information for the model training.
4) The effectiveness of updating pseudo labels and the server update In comparison to the traditional federated learning, our framework can update the global network with the synthetic data at the server, which has the potential to further improve the performance. The results in Fig.5 show that the model performance on FashionMNIST will drop nearly 3% without any server update. However, too many server updates also degrade the performance because too much involvement of the synthetic data will relieve the effects of the local update. The empirical results show that the model trained solely with the synthetic data (the∞ steps) can only get 66.0% accuracy, which highlights the necessity to effectively utilize the synthetic data as well as the local data for model training.
Besides, keeping updating pseudo labels for the synthetic data improves the model performance. As illustrated in Fig.6, the accuracy gradually increases when continuously updating the pseudo labels,
which demonstrates that our framework can improve the confidence level of the pseudo labels in the training process. Note that since our framework only updates the pseudo labels instead of the samples, the extra communication overhead is neglectable.
5) The ablation studies of computational cost and the number of samples for training the generators We present the results in Fig.7 and Fig.8, which show that insufficient training rounds and training samples for generators result in low-quality synthetic data and degraded performance. The generator only gets an FID score of 217.81 (23.17 for real data) and an accuracy of 83.76% when training the generators with 30 rounds. In addition, as shown in Fig.8, under the fixed 30- round computational cost, using fewer samples for training the generators reduces the quality of the synthetic data and thus affects the performance. Nonetheless, despite the low computational cost (30 rounds) and the small number of samples (1000 samples) used to train the generators individually, our framework still outperforms other baselines (82.41% in Fedmix), demonstrating the robustness of the proposed method.
In addition to the influence of the quality of synthetic data on the performance, other factors, such as the threshold τ , λ2, and the size of the synthetic dataset, also affect the global model. More ablation studies about these factors are shown in Appendix C.

heading: 6 CONCLUSIONS

text: We proposed a new data augmentation method to resolve the heterogeneous data distribution problem in federated learning, by introducing pretrained GANs to construct a differentially private global shared dataset. To effectively utilize the synthetic data, a novel framework, named Synthetic Data Aided Federated Learning (SDA-FL), was developed, which generates and updates confident pseudo labels for the synthetic data samples. By supplementing the non-IID private datasets with highquality synthetic data samples, this framework effectively improves the consistency among local models. Experiment results showed that SDA-FL outperforms many existing baselines by remarkable margins in various image classification tasks and is still functional under strict differential privacy protection.

heading: A EXPERIMENTAL DETAIL

text:
Datasets To demonstrate the generalizability of our framework, we use five widely used datasets to test our frameworks, where the details of the datasets are summarized in Table 2. In all the experiments, we equally divide the training samples and assign them to the clients based on the nonIID setting. Specifically, given the number of classes per client as C and with K clients, the whole dataset will be split into K ∗C subsets, and each subset only has a single class. Then all the subsets of data will be randomly shuffled and distributed to clients. For example, in the MNIST experiment, under the scenario that each client only accesses two classes of data, given K = 10 clients, each
client keeps 60000/10 = 6000 samples, each class with 3000 samples. The data distribution is kept the same in the comparison between our framework and other baselines. Except for the Covid-19 dataset, we set 10 clients for all the experiments, and all the clients participate in the training at every communication round.
Models For fair comparisons, we adopt a simple CNN model that consists of two convolution layers and two fully-connected layers for the MNIST and FashionMNIST classification tasks. Meanwhile, ResNet18 (He et al., 2016) is used for classifying the Cifar-10, SVHN, and Covid-19 image datasets. To generate the qualified synthetic samples, we set a generator with four deconvolution layers and a discriminator with four layers convolution layers followed by a fully-connected layer.
Experimental settings of tasks 1) Federated supervised learning: In the federated supervised and semi-supervised experiments, the parameters always are kept the same in our framework, and they are shown in the Table 3. In the baseline experiments, we set µ = 0.001, 0.01, 0.1, 1.0 for Fedprox, and λ = 0.01, 0.1, 0.2, 0.5 for Naivemix as well as Fedmix. The best performances of them are reported in the Table 1. 2) Federated semi-supervised learning: In this experiment, we compare our framework with other baselines in MNIST, FashionMNIST, and Cifar-10 datasets. Every client only keeps two classes of data, containing 50 labeled instances, and the rest are unlabeled. For fair comparisons, the total gradients steps of SDA-FL are the same as other methods. 3) The trade-off between the privacy budget and performance: We evaluate SDA-FL in MNIST and FashionMNIST with the privacy budget = 5, 10, 20,∞ and 30 training rounds (each round with 90 steps) for GANs. In the Cifar-10 experiment, we set the privacy budget = 50, 100, 200,∞ to train the GANs with 400 training rounds (each round with 90 steps). Every client keeps two classes of data in all experiments. 4) The effectiveness of updating pseudo labels and the server update: In this task, we test our
framework in the FashionMNIST dataset and distribute two classes of data to every client. In our framework, the PS updates the pseudo labels for the synthetic dataset every communication step to maintain the confidence of the synthetic data. Compared with the federated supervised learning, we change to update the pseudo labels in the first 2, 10, 20 communication steps instead of updating the pseudo labels at all the communication steps in our framework. In the ablation study of the server update, we perform the experiments with different steps of server update (0, 20, 50, 100,∞) and batch size B = 64. 5) The ablation studies of computational cost and the number of samples for training the generators: The settings of all the ablation studies are the same as the federated supervised learning, and we test them in FashionMNIST with 2 classes of data at each client. All the experimental settings of ablation studies are shown in Table 4.

heading: B THE APPLICATION IN THE COVID-19 DATASET

text:
Because of the scarcity of Pneumonia data, we only set up six clients in this experiment, each of which keeps two classes of data, and the data distribution is shown in Table 5. We test SDA-FL with 30 local steps, 10 server steps, and 50 rounds for training the generators locally. In addition to the baselines in the federated supervised learning task, we also add FedDPGAN (Zhang et al., 2021) for comparison, which trains a global GAN to solve the non-IID issue for medical applications, and the results are shown in Table 6.
Our framework outperforms the other baseline algorithms by a remarkable margin. Particularly, SDA-FL is superior to FedDPGAN by 1.68% in the test accuracy, which indicates that the GANs which are trained individually generate higher-quality synthetic data than the global GAN trained based on the federated learning framework. Furthermore, in addition to addressing the non-IID issue, we also claim that the synthetic data can augment the data distribution, as SDA-FL is superior to FedAvg (IID) by 1.14% in the test accuracy. As a result, we assert that SDA-FL is able to show its advantages of solving non-IID problems and generating high-value data in medical applications.

heading: C ABLATION STUDIES

text:
The size of the synthetic dataset Before the federated learning, all the clients constitute the synthetic dataset together. Larger synthetic datasets can typically provide more qualified data to clients and PS, thus improving performance, but they require larger storage space for clients. The results in Fig.9 show that, although the test accuracy on FashionMNIST decreases as the size of the synthetic dataset declines, our framework achieves 85.44% test accuracy with each client only uploading 200 samples, which is acceptable in exchange for less storage space requirement.
The hyper-parameter λ2 λ2 is the parameter to control the use of the local real data. The results in Fig.10 show that compared with the normal settings of λ2, the test accuracy on FashionMNIST drops by nearly 5.0% with λ2 = 0.25, indicating that the training of the models is heavily reliant on the real data. Besides, an excessive λ2 also degrades the performance since it relieves the effects of the synthetic data and cannot mitigate the non-IID problem as much as possible. In comparison, a moderate λ2 is capable of combining the real data and the synthetic data intelligently and thus enhancing the models and solving the non-IID issue.
Threshold τ Threshold τ sets the criterion for selecting the high-confidence pseudo labels. The results on FashionMNIST are shown in Fig.11, which indicates that a small threshold impairs the performance because some synthetic data with low-confidence pseudo labels are still considered qualified and used to train the models. An excessive threshold, on the other hand, filters out a lot of qualified data, which affects performance. Therefore, τ = 0.95 is a reasonable value to achieve a balance between quality and the quantity of the pseudo labels.


summaries: This paper proposes a loss function for training the base classifier for randomized smoothed classifiers. Specifically, the loss distinguishes between training examples with high prediction confidence and those with low confidence. Several empirical tricks are applied to design the loss function to optimize the performance. Experiments on MNIST and CIFAR-10 show that the proposed training method is superior to existing state-of-the-art randomized smoothed classifiers, especially when the radius r is large.

summaries: This paper proposed new loss functions during the training of classifiers for certified robustness via randomized smoothing. The new loss function treat samples with different confidence level differently. The main idea is to prioritize samples with high confidence because it provides more additional certified radius when its confidence grows.

summaries: This paper proposes CAT-RS which combines two novel losses for training base classifiers for randomized smoothing. The two novel losses aim at preserving clean accuracy for hard samples and improving the certified radius for easy samples. In term of ACR (average certified radius), the method achieves state-of-the-art on MNIST and CIFAR-10.

summaries: The paper studies certified robustness via randomized smoothing (RS). RS has a fundamental accuracy and robustness tradeoff. The authors aim to enhance such tradeoff through a sample-wise control of robustness over the training samples. In particular, the authors investigate the correspondence between robustness and prediction confidence of smoothed classifiers and design a new loss function. The proposed method is evaluated on MNIST and CIFAR10.

abstractText: Any classifier can be “smoothed out” under Gaussian noise to build a new classifier that is provably robust to `2-adversarial perturbations, viz., by averaging its predictions over the noise, namely via randomized smoothing. Under the smoothed classifiers, the fundamental trade-off between accuracy and (adversarial) robustness has been well evidenced in the literature: i.e., increasing the robustness of a classifier for an input can be at the expense of decreased accuracy for some other inputs. In this paper, we propose a simple training method leveraging this trade-off for obtaining more robust smoothed classifiers, in particular, through a sample-wise control of robustness over the training samples. We enable this control feasible by investigating the correspondence between robustness and prediction confidence of smoothed classifiers: specifically, we propose to use the “accuracy under Gaussian noise” as an easy-to-compute proxy of adversarial robustness for each input. We differentiate the training objective depending on this proxy to filter out samples that are unlikely to benefit from the worst-case (adversarial) objective. Our experiments following the standard benchmarks consistently show that the proposed method, despite its simplicity, exhibits improved certified robustness upon existing state-of-the-art training methods.

authors:
Dario Amodei
Chris Olah
Jacob Steinhardt
Paul Christiano
John Schulman
Dan Mané

title: Concrete problems in AI safety

venue: arXiv preprint arXiv:1606.06565,

authors:
Anish Athalye
Nicholas Carlini
David Wagner

title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

venue: In Proceedings of the 35th International Conference on Machine Learning,

authors:
Anand Bhattad
Min Jin Chong
Kaizhao Liang
Bo Li
D.A. Forsyth

title: Unrestricted adversarial examples via semantic manipulation

venue: In International Conference on Learning Representations,

authors:
dlish
Alec Radford
Ilya Sutskever
Dario Amodei

title: Language models are few-shot learners

venue: Advances in Neural Information Processing Systems,

authors:
Nicholas Carlini
David Wagner

title: Towards evaluating the robustness of neural networks

venue: In IEEE Symposium on Security and Privacy,

authors:
Nicholas Carlini
Anish Athalye
Nicolas Papernot
Wieland Brendel
Jonas Rauber
Dimitris Tsipras
Ian Goodfellow
Aleksander Madry

title: On evaluating adversarial robustness

authors:
Rich Caruana
Yin Lou
Johannes Gehrke
Paul Koch
Marc Sturm
Noemie Elhadad

title: Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission

venue: In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,

authors:
Ting Chen
Simon Kornblith
Mohammad Norouzi
Geoffrey Hinton

title: A simple framework for contrastive learning of visual representations

venue: Proceedings of the 37th International Conference on Machine Learning,

authors:
Jeremy Cohen
Elan Rosenfeld
Zico Kolter

title: Certified adversarial robustness via randomized smoothing

venue: In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning,

authors:
Chuan Guo
Geoff Pleiss
Yu Sun
Kilian Q. Weinberger

title: On calibration of modern neural networks

venue: Proceedings of the 34th International Conference on Machine Learning,

authors:
Kaiming He
Xiangyu Zhang
Shaoqing Ren
Jian Sun

title: Deep residual learning for image recognition

venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,

authors:
Dan Hendrycks
Steven Basart
Norman Mu
Saurav Kadavath
Frank Wang
Evan Dorundo
Rahul Desai
Tyler Zhu
Samyak Parajuli
Mike Guo

title: The many faces of robustness: A critical analysis of out-of-distribution generalization

venue: arXiv preprint arXiv:2006.16241,

authors:
Jongheon Jeong
Jinwoo Shin

title: Consistency regularization for certified robustness of smoothed classifiers

authors:
Jongheon Jeong
Sejun Park
Minkyu Kim
Heung-Chang Lee
Doguk Kim
Jinwoo Shin

title: SmoothMix: Training confidence-calibrated smoothed classifiers for certified adversarial robustness

venue: In ICML 2021 Workshop on Adversarial Machine Learning,

title: Learning multiple layers of features from tiny images

venue: Technical report,

authors:
Y. LeCun
L. Bottou
Y. Bengio
P. Haffner

title: Gradient-based learning applied to document recognition

venue: Proceedings of the IEEE,

authors:
Mathias Lecuyer
Vaggelis Atlidakis
Roxana Geambasu
Daniel Hsu
Suman Jana

title: Certified robustness to adversarial examples with differential privacy

venue: IEEE Symposium on Security and Privacy (SP),

authors:
Kimin Lee
Honglak Lee
Kibok Lee
Jinwoo Shin

title: Training confidence-calibrated classifiers for detecting out-of-distribution samples

authors:
Bai Li
Changyou Chen
Wenlin Wang
Lawrence Carin

title: Certified adversarial robustness with additive noise

venue: In Advances in Neural Information Processing Systems

authors:
Aleksander Madry
Aleksandar Makelov
Ludwig Schmidt
Dimitris Tsipras
Adrian Vladu

title: Towards deep learning models resistant to adversarial attacks

authors:
Seyed-Mohsen Moosavi-Dezfooli
Alhussein Fawzi
Pascal Frossard

title: DeepFool: a simple and accurate method to fool deep neural networks

authors:
Hadi Salman
Jerry Li
Ilya Razenshteyn
Pengchuan Zhang
Huan Zhang
Sebastien Bubeck
Greg Yang

title: Provably robust deep learning via adversarially trained smoothed classifiers

authors:
Hadi Salman
Andrew Ilyas
Logan Engstrom
Ashish Kapoor
Aleksander Madry

title: Do adversarially robust imagenet models transfer better

venue: In ArXiv preprint arXiv:2007.08489,

authors:
David Silver
Julian Schrittwieser
Karen Simonyan
Ioannis Antonoglou
Aja Huang
Arthur Guez
Thomas Hubert
Lucas Baker
Matthew Lai
Adrian Bolton

title: Mastering the game of go without human knowledge

authors:
Christian Szegedy
Wojciech Zaremba
Ilya Sutskever
Joan Bruna
Dumitru Erhan
Ian Goodfellow
Rob Fergus

title: Intriguing properties of neural networks

authors:
Florian Tramer
Nicholas Carlini
Wieland Brendel
Aleksander Madry

title: On adaptive attacks to adversarial example defenses

venue: In Advances in Neural Information Processing Systems,

authors:
Dimitris Tsipras
Shibani Santurkar
Logan Engstrom
Alexander Turner
Aleksander Madry

title: Robustness may be at odds with accuracy

authors:
Ashish Vaswani
Noam Shazeer
Niki Parmar
Jakob Uszkoreit
Llion Jones
Aidan N Gomez
Łukasz Kaiser
Illia Polosukhin

title: Attention is all you need

authors:
Oriol Vinyals
Igor Babuschkin
Wojciech M Czarnecki
Michaël Mathieu
Andrew Dudzik
Junyoung Chung
David H Choi
Richard Powell
Timo Ewalds
Petko Georgiev

title: Grandmaster level in starcraft ii using multi-agent reinforcement learning

authors:
Yisen Wang
Difan Zou
Jinfeng Yi
James Bailey
Xingjun Ma
Quanquan Gu

title: Improving adversarial robustness requires revisiting misclassified examples

authors:
Eric Wong
Zico Kolter

title: Provable defenses against adversarial examples via the convex outer adversarial polytope

authors:
Dongxian Wu
Shu-Tao Xia
Yisen Wang

title: Adversarial weight perturbation helps robust generalization

authors:
Kai Y. Xiao
Vincent Tjeng
Nur Muhammad (Mahi) Shafiullah
Aleksander Madry

title: Training for faster adversarial robustness verification via inducing ReLU stability

authors:
Ekim Yurtsever
Jacob Lambert
Alexander Carballo
Kazuya Takeda

title: A survey of autonomous driving: Common practices and emerging technologies

venue: IEEE Access,

authors:
Runtian Zhai
Chen Dan
Di He
Huan Zhang
Boqing Gong
Pradeep Ravikumar
Cho-Jui Hsieh
Liwei Wang

title: MACER: Attack-free and scalable robust training via maximizing certified radius

authors:
Hongyang Zhang
Yaodong Yu
Jiantao Jiao
Eric Xing
Laurent El Ghaoui
Michael Jordan

title: Theoretically principled trade-off between robustness and accuracy

venue: In Proceedings of the 36th International Conference on Machine Learning,

authors:
Huan Zhang
Hongge Chen
Chaowei Xiao
Sven Gowal
Robert Stanforth
Bo Li
Duane Boning
Cho-Jui Hsieh

title: Towards stable and efficient training of verifiably robust neural networks

authors:
Jingfeng Zhang
Xilie Xu
Bo Han
Gang Niu
Lizhen Cui
Masashi Sugiyama
Mohan Kankanhalli

title: Attacks which do not kill training make adversarial learning stronger

authors:
2019 Salman et al
2020 Zhai et al
Jeong
2020 Shin

title: 2021). We also normalize the images in pixel-wise by the mean and the standard deviation calculated from the training set. The full dataset can be downloaded at https: //www.cs.toronto.edu/ ̃kriz/cifar.html

venue: HYPERPARAMETERS

heading: 1 INTRODUCTION

text:
Despite these tremendous advances in deep neural networks for a variety of tasks towards artificial intelligence, e.g., visual recognition (He et al., 2016; Chen et al., 2020), natural language processing (Vaswani et al., 2017; Brown et al., 2020), and reinforcement learning (Silver et al., 2017; Vinyals et al., 2019), the broad existence of adversarial examples (Szegedy et al., 2014) is still one of the most significant aspects that reveals the gap between machine learning systems and humans: for a given input x (e.g., an image) to a classifier f , say a neural network, f often permits a perturbation δ that completely flips the prediction f(x+ δ), while δ is too small to change the semantic in x. In response to this vulnerability, there have been significant efforts in building robust neural network based classifiers against adversarial examples, either in forms of empirical defenses (Athalye et al., 2018; Carlini et al., 2019; Tramer et al., 2020), which are largely based on adversarial training (Madry et al., 2018; Zhang et al., 2019; Wang et al., 2020; Zhang et al., 2020b; Wu et al., 2020), or certified defenses (Wong & Kolter, 2018; Xiao et al., 2019; Cohen et al., 2019; Zhang et al., 2020a), depending on whether the robustness claim can be theoretically guaranteed or not.
Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019), our focus in this paper, is currently a prominent approach in the context of certified defense, thanks to its scalability to arbitrary neural network architectures while previous methods have been mostly limited in network sizes or require strong assumptions on their architectures: specifically, for a given classifier f , it constructs a new classifier f̂ , where f̂(x) is defined to be the class that f(x + δ) outputs most likely over δ ∼ N (0, σ2I), i.e., the Gaussian noise. Then, it is shown by Lecuyer et al. (2019) that f̂ is certifiably robust in `2-norm, and Cohen et al. (2019) further tightened the `2-robustness guarantee which is currently considered as the state-of-the-art in certified defense.
However, even with recent methods for adversarial defense, including randomized smoothing, the trade-off between robustness and accuracy (Tsipras et al., 2019; Zhang et al., 2019) has been well evidenced, i.e., increasing the robustness for a specific input can be at the expense of decreased accuracy for other inputs. For instance, with the current best practices, Salman et al. (2020) report
that the accuracy of ResNet-50 on ImageNet degrades, e.g., 75.8%→ 63.9%, by an `∞-adversarial training, i.e., optimizing the classifier to ensure robustness at all the given training samples around an `∞-ball (of size 4255 ). In addition, Zhang et al. (2019) have shown that the (empirical) robustness of a classifier can be further boosted in training by paying more expense in accuracy. A similar trend can be also observed with certified defenses, e.g., randomized smoothing, as the clean accuracy of smoothed classifiers are usually less than those one can obtain from the standard training on the same architecture (Cohen et al., 2019).
Contribution. In this paper, we develop a novel training method for randomized smoothing, coined Confidence-Aware Training for Randomized Smoothing (CAT-RS), which incorporates a sample-wise control of target robustness on-the-fly motivated by the accuracy-robustness trade-off in smoothed classifiers. Intuitively, a natural approach one can consider in response to the trade-off in robust training is to appropriately lower the robustness requirement for “hard-to-classify” samples while maintaining those for the remaining (“easier”) samples: here, the challenges are (a) which samples we should choose for it during training and (b) how to control their target robustness.
To implement this idea, we focus on the direct correspondence from prediction confidence to adversarial robustness that smoothed classifiers offer: due to its local-Lipschitzness (Salman et al., 2019), achieving a high confidence at x from a smoothed classifier also implies a high (certified) robustness at x. Inspired by this, we propose to use the sample-wise confidence (of smoothed classifiers) as an efficient proxy of the certified robustness, and defines two new losses, namely bottom-K and worstcase Gaussian training, each of those targets different levels of confidence so that the overall training can be more informed sample-wise for better robustness by preventing low-confident samples from being enforced to increase their robustness.
We verify the effectiveness of our proposed method through an extensive comparison with existing robust training methods for smoothed classifiers, including the state-of-the-arts, on a wide range of established benchmarks on MNIST and CIFAR-10 datasets. Our experimental results constantly show that the proposed method can significantly improve the previous state-of-the-art results on certified robustness achievable from a given neural network architecture, by (a) maximizing the robust radii of high-confidence samples while (b) reducing the risk of deteriorating the accuracy at low-confidence samples. Our extensive ablation study further confirms that each of both proposed components has an individual effect on improving certified robustness, and can effectively control the accuracy-robustness trade-off with the hyperparameter between the two proposed losses.

heading: 2 PRELIMINARIES

text:
Adversarial robustness. Consider a labeled dataset D = {(xi, yi)}ni=1 from a certain distribution P , where x ∈ Rd and y ∈ Y := {1, · · · ,K}, which forms a classification problem with K classes. Let f : Rd → Y be a classifier. Notice that f is a discrete and non-differentiable, so that one can additionally consider a differentiable F : Rd → ∆K−1 to allow a gradient-based optimization assuming f(x) := arg maxk∈Y Fk(x), where ∆
K−1 is probability simplex in RK . The standard framework of empirical risk minimization to optimize f assumes that the samples in D are i.i.d. from P and expect f to perform well given that the future samples also follow the i.i.d. assumption.
However, in the context of adversarial robustness (and for other notions of robustness as well), the i.i.d. assumption on the future samples does not hold anymore: instead, it additionally assumes that the samples can be arbitrarily perturbed up to a certain restriction, e.g., a bounded `2-ball, and focuses on the worst-case performance over the perturbed samples. One possible way to quantify this scenario is to consider the average minimum-distance of adversarial perturbation (MoosaviDezfooli et al., 2016; Carlini & Wagner, 2017; Carlini et al., 2019), namely:
R(f ;P ) := E(x,y)∼P [
min f(x′)6=y
||x′ − x||2 ] . (1)
In this respect, we aim to find f that maximizes R(f ;P ) while maintaining the performance on P .
Randomized smoothing. The essential challenge in achieving adversarial robustness in neural networks, however, stems from that directly evaluating (1) (and further optimizing it) is usually computationally infeasible, e.g., under the standard practice that F is modeled by a complex, highdimensional neural network. Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019)
bypasses this difficulty by constructing a new classifier f̂ from f instead of letting f to directly model the robustness: specifically, it transforms the base classifier f with a certain smoothing measure, where in this paper we focus on the case of Gaussian distributions N (0, σ2I):
f̂(x) := arg max c∈Y Pδ∼N (0,σ2I) (f(x+ δ) = c) . (2)
Then, the robustness of f̂ at (x, y), namely R(f̂ ;x, y), can be explicitly lower-bounded in terms of the certified radius R(f̂ , x, y), e.g., Cohen et al. (2019) showed that the following bound holds which is tight for `2-adversary, e.g., it is the optimal for linear classifiers:
R(f̂ ;x, y) ≥ σ · Φ−1(pf (x, y)) =: R(f̂ , x, y) (3) where pf (x, y) := Pδ(f(x+ δ) = y), (4)
provided that f̂(x) = y, otherwise R(f̂ ;x, y) := 0.1 Here, we remark that the formula for certified radius (3) is essentially a function of pf (4), which represents the prediction confidence of f̂ at x, or equivalently, the accuracy of f(x + δ) over δ ∼ N (0, σ2I). In other words, unlike standard neural networks, smoothed classifiers can guarantee a correspondence from prediction confidence to adversarial robustness - which is the key motivation of our method in this paper. Figure 1 plots this relationship shown by Cohen et al. (2019) as well as by some prior works (Lecuyer et al., 2019; Li et al., 2019) which also attempt to lower-bound the robustness of smoothed classifiers.

heading: 3 CONFIDENCE-AWARE TRAINING FOR RANDOMIZED SMOOTHING

text:
We aim to develop a new training method to maximize the certified robustness of f̂ , considering the trade-off relationship between robustness and accuracy (Zhang et al., 2019): even though randomized smoothing can be applied for any classifier f , the actual robustness of f̂ depends how much f classifies well under presence of Gaussian noise, i.e., by pf (x, y) as in (3). A simple way to train f for a robust f̂ , therefore, is to minimize the standard cross-entropy loss (denoted by CE below) with Gaussian augmentation as in Cohen et al. (2019):
min F E (x,y)∼P δ∼N (0,σ2I) [CE(F (x+ δ), y)] . (5)
In this paper, we extend this basic form of training to incorporate a confidence-aware strategy to decide which noise samples δ ∼ N (0, σ2I) should be focused on sample-wise during training of f . Recall Figure 1 that plots mappings from the (smoothed) confidence pf (4) to a certified radius, e.g., those derived by Cohen et al. (2019) (at the blue line). Ideally, one may wish to obtain a classifier f that achieves pf (x, y) ≈ 1 for every (x, y) ∼ P to maximize its certified robustness. In practice, however, such a case is highly unlikely, and there usually exists a sample x that pf (x, y) should be quite lower than 1 to maintain the discriminativity with other samples: in other words, these
1Φ denotes the cumulative distribution function of the standard normal distribution.
samples can be actually “beneficial” to be misclassified at some (hard) Gaussian noises, otherwise the classifier has to memorize the noises to correctly classify them. On the other hand, for the samples which can indeed achieve pf (x, y) ≈ 1, the current Gaussian training in (5) may not be able to provide enough samples of δ for x throughout the training, as pf (x, y) ≈ 1 implies that f(x+ δ) must be correctly classified for “almost every” possibility of δ ∼ N (0, σ2I). Also, considering that the radius certifiable at x rapidly increases as pf (x, y)→ 1 as shown in Figure 1, it is important for an overall robustness of f̂ to increase pf (x, y), especially when it can be close to 1 at the end.
In these respects, we propose two different variants of Gaussian training (5) that address each of the possible cases, i.e., whether (a) pf (x, y) < 1 or (b) pf (x, y) ≈ 1, namely with (a) bottom-K and (b) worst-case Gaussian training, respectively. During training, the method first estimates pf (x, y) for each sample by simply computing their accuracy over M random samples of δ ∼ N (0, σ2I), and applies different forms of loss depending on the value. In the following two sections, i.e., Section 3.1 and 3.2, we provide the details on each of the proposed losses, and Section 3.3 describes how to combine the two losses and defines the overall training scheme.
3.1 BOTTOM-K GAUSSIAN TRAINING: LOSS FOR LOW-CONFIDENCE SAMPLES
Consider a base classifier f and a training sample (x, y) ∈ D, and suppose that pf (x, y) = p 1, e.g., f̂ has a low-confidence at x. Figure 2(a) visualizes this scenario: in this case, by definition of pf (x, y) in (4), f(x + δ) would be correctly classified to y only with probability p over δ, and this can imply either that (a) x + δ has not yet been adequately exposed to f during the training, or (b) x+ δ may be indeed hard to be correctly classified for some noise samples δ, so that minimizing the training loss at these noises could harm the classifier. The design goal of our proposed bottom-K Gaussian training is to modify the standard Gaussian training (5) to reduce the optimization burden from (b) while minimally retaining its ability to cover enough noise samples during training for (a).
To this end, we first consider M random i.i.d. samples of δ, namely δ1, δ2, · · · , δM ∼ N (0, σ2I). Then, one can notice that the random variables 1[f(x + δi) = y]’s are also i.i.d., each of which follows the Bernoulli distribution of probability p, given that pf (x, y) = p. This means that, if the current pf (x, y) is the value one attempts to keep instead of further increasing it, the number of noise samples that should be correctly classified, which can be defined as ∑ i 1[f(x + δi) = y], would follow the binomial distribution, namely K ∼ Bin(M,p), and this motivates us to consider the following loss that only minimizes the K-smallest cross-entropy losses out of from M samples:
Llow := 1
M K∑ i=1 CE(F (x+ δπ(i)), y), where K ∼ Bin(M,pf (x, y)). (6)
Here, π(i) denotes the noise index with the i-th smallest loss value in the M samples.
Yet, the loss defined in (6) may not handle the cold-start problem on pf (x, y), e.g., at the early stage of the training where x + δ has not been adequately exposed to f , so that Llow can be minimized too early with an under-estimated p̂f . We found that, however, a simple trick of clamping K with 1 can bypass the issue: i.e., we always allow the “easiest” noise among the M samples to be fed into f throughout the training. In addition, we also found that it is still beneficial to constrain the samples that are not within K, i.e., the “harder” ones, to ensure that they still have a certain level of confidence for the class y even if they are misclassified: here, as a simple design, we propose to truncate the cross entropy loss with log p0 so that Fy(x+ δ) can be at least p0, e.g., p0 = 120 as done in our experiments. In these respects, we re-define the loss in (6) as in the followings:
Llow := 1
M K+∑ i=1 CE(F (x+ δπ(i)), y) + M∑ i=K++1 [ CE(F (x+ δπ(i)), y) + log p0 ]+ , (7) where K ∼ Bin(M,p), K+ := max(1,K), [·]+ := max(0, ·), and p0 ∈ (0, 1] is a hyperparameter. Note that the loss becomes the standard Gaussian training as p0 → 1 and returns to (6) as p0 → 0.

heading: 3.2 WORST-CASE GAUSSIAN TRAINING: LOSS FOR HIGH-CONFIDENCE SAMPLES

text:
Next, we focus on the case when pf (x, y) ≈ 1, i.e., f̂ has a high confidence at x, as illustrated in Figure 2(b). In contrast to the previous scenario in Section 3.1 (and Figure 2(a)), now the major
drawback of Gaussian training (5) does not come from the abundance of hard noises during training, but from the sparseness of such noises: considering that one can only present a limited number of noise samples to f throughout its training, naı̈vely minimizing (5) may not cover some “potentially hard” noise samples, and this would result in a significant harm in certified radius especially at the regime of pf (x, y) ≈ 1 as shown in Figure 1. The purpose of worst-case Gaussian training is to overcome this lack of samples via an adversarial search around each of the noise samples.
Specifically, given that we have M samples of δ as in (7), namely δ1, δ2, · · · , δM ∼ N (0, σ2I), we propose to modify (5) to find and minimize the worst-case noise (a) around an `2-ball for each noise as well as (b) among the M samples, instead of minimizing the average-case loss:
Lhigh := max i max ‖δ∗i−δi‖2≤ε CE(F (x+ δ∗i ), y). (8)
We use the projected gradient descent (PGD) (Madry et al., 2018) to solve the inner maximization in (8): namely, we perform a T -step gradient ascent from each δi with step size 2 · ε/T while projecting the perturbations to be in the `2-ball of size ε. Here, although T and ε can be hyperparameters that affect the inner maximization, we simply fix them in our experiments by ε = 1.0 and T = 4. We also make sure that the likelihood of the optimized δ∗ as i.i.d. Gaussian still remains high by simply normalizing the mean and standard deviation of δ∗ to follow those of the original δ.
Comparison to SmoothAdv. The idea of incorporating an adversarial search for the robustness of smoothed classifiers has been also considered in previous works (Salman et al., 2019; Jeong et al., 2021): e.g., Salman et al. (2019) have proposed SmoothAdv that applies adversarial training (Madry et al., 2018) to a “soft” approximation of f̂ given f and M noise samples:
x∗ = arg max ||x′−x||2≤
( − log ( 1
M ∑ i Fy(x ′ + δi)
)) . (9)
Our method is different from the previous approaches in which part of the inputs is adversarially optimized: i.e., we directly optimize the noise samples δi’s instead of x, with no need to assume a soft relaxation of f̂ . This is due to our unique motivation of finding the worst-case Gaussian noise, and our experimental results in Section 4 further support the effectiveness of this approach.

heading: 3.3 OVERALL TRAINING SCHEME

text:
Given the two losses Llow and Lhigh defined in Section 3.1 and 3.2, respectively, we now define the full objective of our proposed Confidence-Aware Training for Randomized Smoothing (CATRS). Overall, in order to differentiate how to combine the two losses per sample basis, we use the smoothed confidence pf (x, y) (4) as the guiding proxy: specifically, we aim to apply the worst-case loss of Lhigh only for the samples where pf (x, y) is already high enough. In practice, however, one does not have a direct access to the value of pf (x, y) during training, and we estimate this with the M noise samples2 as done for Llow and Lhigh, i.e., by p̂f (x, y) := 1M ∑M i=1 1[f(x+δi) = y]. Here, we set a simple condition of “p̂f (x, y) = 1” to activate Lhigh, and the final loss becomes:
LCAT-RS := Llow + λ · 1[p̂f (x, y) = 1] · Lhigh, (10)
where 1[·] is the indicator random variable, and λ > 0 is a hyperparameter. The complete procedure of computing our proposed CAT-RS loss can be found in Appendix A.

heading: 4 EXPERIMENTS

text:
In this section, we evaluate the effectiveness of our proposed training scheme compared to existing state-of-the-art training methods for smoothed classifiers, based on two well-established benchmarks on MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky, 2009) extensively in compliance to the standard protocol of the previous works (Cohen et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021). Overall, the experiments show that our method can consistently outperform the previous best efforts to improve the average certified radius by (a) maximizing the robust radii of high-confidence samples while (b) better maintaining the accuracy at low-confidence samples. We
2In our experiments, we use M = 4 for our method unless otherwise noted.
also perform a thorough ablation study on our loss design and confirm that (a) both of our proposed components have effectiveness to improve the certified robustness, and (b) the single hyperprameter λ (10) between the two losses can balance the trade-off between robustness and accuracy.
Training setups. For a fair comparison, we follow the training setup considered in most of the previous works to compare the performance of the smoothed classifiers (Cohen et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021): specifically, we mainly consider LeNet (LeCun et al., 1998) and ResNet-110 (He et al., 2016) for MNIST and CIFAR-10, respectively, and consider three different scenarios of σ = 0.25, 0.5, and 1.0 for randomized smoothing. We train each model for 150 epochs with stochastic gradient descent (SGD) with a momentum of 0.9. The learning rates are initialized to 0.01 for MNIST and 0.1 for CIFAR-10, and decreased by a factor of 0.1 in every 50 epochs. We apply the same σ for both training and evaluation. More details on the training setups, e.g., hyperparameters, can be found in Appendix B.
Baselines. We compare our method with an extensive list of baseline methods in the literature for training smoothed classifiers: (a) Gaussian training (Cohen et al., 2019) simply trains a classifier with Gaussian augmentation (5); (b) Stability training (Li et al., 2019) adds a cross-entropy term between the logits from clean and noisy images; SmoothAdv (Salman et al., 2019) employs adversarial training for smoothed classifiers (9); (d) MACER (Zhai et al., 2020) adds a regularization that aims to maximize a soft approximation of certified radius; (e) Consistency (Jeong & Shin, 2020) regularizes the variance of confidences over Gaussian noise; (f) SmoothMix (Jeong et al., 2021) proposes a mixup-based (Zhang et al., 2018) adversarial training for smoothed classifiers. Whenever possible, we use the pre-trained models publicly released by the authors to reproduce the results.
Evaluation metrics. We follow the standard evaluation protocol for smoothed classifiers (Salman et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021): specifically, Cohen et al. (2019) have proposed a practical Monte Carlo based certification procedure, namely CERTIFY, that returns the prediction of f̂ and a lower bound of certified radius, CR(f, σ, x), over the randomness of n samples with probability at least 1 − α, or abstains the certification. Based on CERTIFY, we consider two major evaluation metrics: (a) the average certified radius (ACR) (Zhai et al., 2020): the average of certified radii on the test datasetDtest while assigning incorrect samples as 0, namely ACR := 1|Dtest| ∑ (x,y)∈Dtest [CR(f, σ, x) ·1f̂(x)=y], and (b) the approximate certified test accuracy at r: the fraction of the test dataset which CERTIFY classifies correctly with the radius larger than r without abstaining. We use n = 100, 000, n0 = 100, and α = 0.001 for CERTIFY, following the previous works (Cohen et al., 2019; Salman et al., 2019; Jeong & Shin, 2020; Jeong et al., 2021).

heading: 4.1 RESULTS ON MNIST

text:
We compare the certified robustness of the smoothed classifiers on MNIST from our method to those from other baselines in Table 1. We also present in Figure 3 the plots of the approximate certified accuracy across varying r for σ ∈ {0.25, 0.5, 1.0}. Overall, the results show that our method of CAT-RS clearly surpasses all the other baselines in terms of ACR: i.e., our method could better balance between the clean accuracy and robustness. For σ = 0.25, we notice that some baselines, i.e., SmoothAdv and SmoothMix, already achieve a reasonably saturated level of ACR: even in this trivial task, our method could further improve the robust accuracy at r = 0.75 as 96.8% → 97.0%. In more challenging cases of σ = 0.5 and σ = 1.0, on the other hand, the improvements from CATRS in ACR become more evident as σ increases: e.g., at σ = 1.0, compared to SmoothMix (the bestperforming baseline), CAT-RS could improve the certified accuracy at r = 2.50 by 28.9%→ 29.8% while even improving the clean accuracy (i.e., certified accuracy at r = 0.0) by 93.7% → 93.9%. This means that our proposed CAT-RS can be more effective at challenging tasks, where it is more likely that a given classifier gets a more diverse confidence distribution for the training samples, so that our proposed confidence-aware training can better play its role.
Accuracy-robustness trade-off. To further validate that our method can exhibit a better trade-off between accuracy and robustness compared to other methods, we additionally compare the performance trends between clean accuracy and certified accuracy at r = 2.0 as we vary a hyperparameter to control the trade-off, e.g., λ (10) in case of our method. We use σ = 1.0 for this experiment. We choose Consistency (Jeong & Shin, 2020) and SmoothMix (Jeong et al., 2021) for this comparison, considering that they also offer a single hyperparameter (namely λ and η, respectively) for the balance between accuracy and robustness similar to our method, while both generally achieve good performances among the baselines considered. The results plotted in Figure 5 clearly show that CAT-RS indeed exhibits a higher trade-off frontier compared to both methods, which confirms the effectiveness of our method. More detailed results can be found in Appendix C.

heading: 4.2 RESULTS ON CIFAR-10

text:
Table 2 shows the performance of the baselines and our model on CIFAR-10. In Figure 4, we also plot the approximate certified accuracy over the range of r for σ ∈ {0.25, 0.5, 1.0}. In this experiment, for each σ, the baseline models are individually chosen based on its “best” ACR, i.e.,
Consistency SmoothMix CAT-RS (ours)Ce rti fie
d Ac
c. @
R =
2 .0
(% )
35
40
45
50
Clean Acc. (%) 92 93 94 95 96 97
Figure 5: Comparison of clean vs. certified accuracy at r = 2.0 on MNIST (σ = 1.0) for methods those offer trade-off control.
1/p0 = 1 1/p0 = 2 1/p0 = 4 1/p0 = 8 1/p0 = 16 1/p0 = 32 1/p0 = ∞
C er
tif ie
d ac
cu ra
cy
0
0.2
0.4
0.6
Radius 0 0.5 1.0 1.5 2.0
(a) Effect of p0
λ= 0.25 λ= 0.50 λ= 1.00 λ= 2.00 λ= 4.00
C er
tif ie
d ac
cu ra
cy
0
0.2
0.4
0.6
Radius 0 0.5 1.0 1.5 2.0
(b) Effect of λ
Figure 6: Comparison of certified accuracy of CAT-RS ablations on CIFAR-10. We use ResNet-20 for ablation study and plot the results at σ = 0.5. More results can be found in Appendix D.
the hyperparameter of the same baseline may vary over σ. For example, we choose the SmoothAdv baseline as the best model from the hundreds of hyperparameter configurations those examined by Salman et al. (2019) for each σ. Overall, our method of CAT-RS achieves a significant improvement of ACR compared to the baselines. In case of σ = 0.25 and σ = 0.5, for example, CAT-RS clearly offers a better trade-off between the clean accuracy and robustness compared to SmoothAdv, in a sense that (a) they achieve similar certified accuracy at large r, yet (b) CAT-RS could maintain much higher clean accuracy, e.g., 53.2% → 60.2% in case of σ = 0.5. For σ = 1.0, the ACR of our method significantly surpasses the previous best model, SmoothMix, by 0.773 → 0.812. As in MNIST, the improvement of CAT-RS is most evident in σ = 1.0, demonstrating the effectiveness of confidence-aware training. Furthermore, we observe that the performance gap between our method and the previous best model is more significant in CIFAR-10 because the higher complexity of CIFAR-10 compared to MNIST makes the confidence information more critical.

heading: 4.3 ABLATION STUDY

text:
We also conduct an ablation study to further analyze individual effectiveness of the design components in our method. Unless otherwise specified, we use ResNet-20 (He et al., 2016) throughout this section and test it on a subsampled CIFAR-10 of size 500. We assume λ = 1.0 and p0 = 110 by default. We report the detailed results for this study in Appendix D.
Effect of p0. We introduce a hyperparameter p0 in (7) to control how much we constrain the confidence of “hard” noises, so that p0 → 1 would lead the proposed Llow (7) into the standard
Gaussian training. To verify its effectiveness, we conduct an experiment comparing the certified robustness of models with different p0 ∈ {0, 132 , 1 16 , 1 8 , 1 4 , 1 2 , 1} on σ ∈ {0.25, 0.5, 1.0}: Table 5 in Appendix D summarizes the results. Overall, we observe that setting p0 < 1 is clearly beneficial to improve ACR for all the σ considered, e.g., in case of σ = 1.0 the method achieve the best ACR when p0 = 116 , 1 32 , and this confirms that our proposed loss of L
low (7) alone is superior to the Gaussian training even without the effect from Lhigh (8). From the Figure 6(a) which plots the results at σ = 0.5, one can observe that letting p0 → 0 leads the classifier to achieve a better robust accuracy with a relatively little degradation in the clean accuracy.
Effect of λ. By the definition in (10), λ controls the contribution of Lhigh. We evaluate the impact of λ here, and the results are shown in Figure 6(b). We compare the performance of the models, varying λ ∈ {0.25, 0.5, 1.0, 2.0, 4.0} on σ = 0.5. As expected, we observe that λ balances the trade-off between robustness and clean accuracy; as λ increases, robustness increases while clean accuracy decreases. Also, in Table 7 in Appendix C, we verify that CAT-RS offers more effective trade-off between robustness and the clean accuracy than other methods. Further details can be found in Table 6 in Appendix D.
Loss design. Our loss design in (10) combines several important ideas proposed in Section 3, and here we validate that each of the components has an individual effect in improving the certified robustness. In Table 3, we compare several variants of our proposed CAT-RS loss (10), namely (a) using only Llow (7), (b) using only Lhigh (8), and (c) Llow +λ ·Lhigh, i.e., not applying the masking condition 1[p̂f (x, y) = 1] to Lhigh. We also consider (d) a variant of Lhigh, where the outer max operation over noise samples (8) is replaced by the average, i.e., we define:
Lavg max := 1
M ∑ i ( max ‖δ∗i−δi‖2≤ε CE(F (x+ δ∗i ), y) ) . (11)
We report the experimental results on σ = 0.5. Overall, CAT-RS achieves the best ACR and approximate certified accuracy among the variants. We observe that Llow alone improves Gaussian baseline, which highlights the importance of the confidence information for robust training. Also, Lhigh alone increases the robustness dramatically, while it sacrifices the clean accuracy. Combining Llow and Lhigh achieves better results by balancing robustness and accuracy. CAT-RS further improves the overall performance by simply masking out Lhigh for low-confidential predictions: i.e., when the confidence is not high enough, learning challenging adversarial samples harm the performance. By comparing the results made by using Lavg max and Lhigh, we verify that it is more helpful for a model to utilize the most challenging adversarial examples instead of averaging them.

heading: 5 CONCLUSION

text:
This paper explores a close relationship between confidence and robustness, a natural property of smoothed classifiers yet neural networks cannot currently offer. We have successfully leveraged this relationship to relax the hard-to-compute metric of adversarial robustness into an easier concept of prediction confidence. Consequently, we could propose a practical robust training method that enables a sample-level control of adversarial robustness, which has been difficult without heuristics in a conventional belief. We believe our work could be a useful step for the future research on exploring the interesting connection between adversarial robustness and confidence calibration (Guo et al., 2017; Lee et al., 2018) through the framework of randomized smoothing.
ETHICS STATEMENT
Deploying deep learning based systems into the real-world, especially when they are of securityconcerned (Caruana et al., 2015; Yurtsever et al., 2020), still poses many potential risks for both companies and customers, and we researchers are responsible to make this technology more reliable through research towards AI safety (Amodei et al., 2016). Adversarial robustness is one of the central parts of this direction, and we believe our research on certified robustness can be a useful step towards building a practical yet secure deep learning system. Nevertheless, one should also recognize that adversarial robustness is still a bare minimum requirement for reliable deep learning, and the future research should address how to extend this restrictive notion of robustness into other challenging setups, e.g., corruption robustness (Hendrycks et al., 2020) and unrestricted attacks (Bhattad et al., 2020), just to name a few, to establish a practical sense of security for practitioners.
REPRODUCIBILITY STATEMENT
For the best practices to maintain the reproducibility of this paper, we have conducted our experiments on publicly available datasets, with the detailed descriptions for pre-processing in Appendix B.1. We have documented a thorough specification on the experimental details, e.g., training setups, baselines, and evaluation metrics, in Section 4. We have also fully specified all the hyperparameters considered to reproduce the baselines as well as our results in Appendix B.2. The code to reproduce our results will be publicly available, as well as the pre-trained models for our method.

heading: A TRAINING PROCEDURE OF CAT-RS

text:
Algorithm 1 Confidence-Aware Training for Randomized Smoothing (CAT-RS) Require: training sample (x, y). smoothing factor σ. number of noise samples M > 0. truncation
probability p0 ∈ (0, 1], regularization strength λ > 0. attack norm ε > 0.
1: Sample δ1, · · · , δM ∼ N (0, σ2I) 2: p̂f ← 1M ∑ i 1[f(x+ δi) = y] 3: // COMPUTE THE BOTTOM-K LOSS 4: Sample K ∼ Bin(M, p̂f ) 5: K+ ← max(1,K) 6: for i = 1 to M do 7: Li ← CE(F (x+ δi), y) 8: end for 9: Lπ1:M ← argsort(L1:M )
10: Llow ← 1M ( ∑K+ i=1 L π i + ∑M i=K++1(L π i + log p0)
+) 11: // COMPUTE THE WORST-CASE LOSS 12: for i = 1 to M do 13: δ∗i ← arg max‖δ∗i−δi‖≤εCE(F (x+ δ ∗ i ), y) 14: end for 15: Lhigh ← maxiCE(F (x+ δ∗i ), y) 16: // COMPUTE THE CAT-RS LOSS 17: LCAT-RS ← Llow + λ · 1[p̂f = 1] · Lhigh

heading: B EXPERIMENTAL DETAILS

text:
B.1 DATASETS
MNIST (LeCun et al., 1998) consists of 70,000 gray-scale hand-written digit images of size 28×28, 60,000 for training and 10,000 for testing, where each is labeled to one value between 0 and 9. We do not perform any pre-processing except for normalizing the range of each pixel from 0-255 to 0-1. The dataset can be downloaded at http://yann.lecun.com/exdb/mnist/.
CIFAR-10 (Krizhevsky, 2009) consists of 60,000 RGB images of size 32×32 pixels, 50,000 for training and 10,000 for testing, where each is labeled to one of 10 classes. We use the standard dataaugmentation scheme of random horizontal flip and random translation up to 4 pixels, following the practice of other baselines (Cohen et al., 2019; Salman et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020; Jeong et al., 2021). We also normalize the images in pixel-wise by the mean and the standard deviation calculated from the training set. The full dataset can be downloaded at https: //www.cs.toronto.edu/˜kriz/cifar.html.
B.2 HYPERPARAMETERS
Stability training (Li et al., 2019) uses a single hyperparameter γ to control the relative strength of the regularization term. We fix γ = 2 for MNIST experiments. For CIFAR-10 experiments, γ = 2 is used for σ = 0.25, 0.5, and γ = 1 is used for σ = 1.0.
SmoothAdv (Salman et al., 2019) uses three major hyperparameters to perform the projected gradient descent: namely, the attack radius in terms of `2-norm ε, the number of PGD steps T , and the number of noises m. In our experiments, we fix T = 10. For MNIST experiments, we fix ε = 1.0 and m = 4 as well. In case of CIFAR-10, on the other hand, we report the results chosen among the list of “best” configurations for each noise level which are previously searched by Salman et al. (2019): specifically, we report the results of ε = 1.0 and m = 4 for σ = 0.25, and ε = 2.0 and m = 2 for σ = 0.5, 1.0. When SmoothAdv is used, we adopt the warm-up strategy, i.e., we initially set ε = 0.0 and linearly increase to the target value of ε for 10-epochs.
MACER (Zhai et al., 2020) introduces four hyperparameters: namely, the number of noises k, the coefficient for the regularization term λ, the clamping parameter for maximizing the certified radius γ, and the temperature scaling parameter β. For the MNIST experiments, we use k = 16, γ = 8.0, β = 16.0, and λ = 16.0 when σ = 0.25, 0.5, following the configurations in Zhai et al. (2020). When σ = 1.0, for the training to succeed, we set λ = 6.0. For the CIFAR-10 experiments, we follow the original configurations used by Zhai et al. (2020). We set k = 16, γ = 8.0, and β = 16.0. λ is set to be 12.0 and 4.0 for σ = 0.25 and 0.5, respectively. For σ = 1.0, the training starts with λ = 0 until the first learning rate decay and we set λ = 12.0 thereafter.
Consistency (Jeong & Shin, 2020) uses two hyperparameters: namely, the coefficient for the consistency term η and those for the entropy term γ. We report the best results in terms of ACR among those reported by Jeong & Shin (2020) varying η. Following the original practice, we fix γ = 0.5 throughout our experiments. For MNIST, we use λ = 10 for σ = 0.25 and λ = 5 for other noises. For the CIFAR-10 experiments, we use λ = 20 for σ = 0.25 and λ = 10 for other noises.
SmoothMix (Jeong et al., 2021) introduces four hyperparameters: namely, the coefficient for the mixup loss η, the step size for adversarial attack α, the number of steps for adversarial attack T , and the number of noises T . For the MNIST experiments, we fix η = 5.0, α = 1.0, and m = 4. T = 2, 4, 8 are used for the models with σ = 0.25, 0.5, 1.0, respectively. For the CIFAR-10 experiments, we again report the best result among those reported from Jeong et al. (2021): i.e., we fix η = 5.0,m = 2, and T = 4, and use α = 0.5, 1.0, 2.0 for σ = 0.25, 0.5, 1.0, respectively. The “one-step adversary” is used for σ = 0.5, 1.0 to follow the “best” configurations.
CAT-RS (Ours). CAT-RS uses two main hyperparameters: namely, the coefficient for “worst-case” loss λ and the clamping parameter for maximizing the confidence of “hard” samples p0. For the MNIST experiments, we use the fixed configuration of λ = 1.0 and p0 = 15 . For the CIFAR-10 experiments, we use λ = 0.5, p0 = 120 for σ = 0.25, and λ = 2.0, p0 = 1 10 for σ = 0.5 and 1.0.

heading: C COMPARISON OF ACCURACY-ROBUSTNESS TRADE-OFF

heading: D DETAILED RESULTS ON ABLATION STUDY