Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks (winning tickets) which are capable of reaching accuracy comparable to the original models. However, these tickets are proved to be not robust to adversarial examples, and even worse than their PLM counterparts. To address this problem, we propose a novel method based on learning binary weight masks to identify robust tickets hidden in the original PLMs. Since the loss is not differentiable for the binary mask, we assign the hard concrete distribution to the masks and encourage their sparsity using a smoothing approximation of L 0 regularization. Furthermore, we design an adversarial loss objective to guide the search for robust tickets and ensure that the tickets perform well both in accuracy and robustness. Experimental results show the significant improvement of the proposed method over previous work on adversarial robustness evaluation. Large-scale pre-trained language models (PLMs), such as BERT (Devlin et al., 2019), Roberta (Liu et al., 2019) and T5 (Raffel et al., 2019) have achieved great success in the field of natural language processing. As more transformer layers are stacked with larger self-attention blocks, the complexity of PLMs increases rapidly. Due to the over-parametrization of PLMs, some Transformer heads and even layers can be pruned without significant losses in performance (Michel et al., 2019; Kovaleva et al., 2019; Rogers et al., 2020). The Lottery Ticket Hypothesis suggests an over-parameterized network contains certain subnetworks (i.e., winning tickets) that can match the performance of the original model when trained in isolation (Frankle and Carbin, 2019). Chen  Equal contribution. et al. (2020); Prasanna et al. (2020) also find these winning tickets exist in PLMs. Chen et al. (2020) prune BERT in an unstructured fashion and obtain winning tickets at sparsity from 40% to 90%. Prasanna et al. (2020) aim at finding structurally sparse tickets for BERT by pruning entire attention heads and MLP. Previous works mainly focused on using winning tickets to reduce model size and speed up training time (Chen et al., 2021), while little work has been done to explore more benefits, such as better adversarial robustness than the original model. As we all know, PLMs are vulnerable to adversarial examples that are legitimately crafted by imposing imperceptible perturbations on normal examples (Jin et al., 2020; Garg and Ramakrishnan, 2020; Wang et al., 2021). Recent studies have shown that pruned subnetworks of PLMs are even less robust than their PLM counterparts (Xu et al., 2021; Du et al., 2021). Xu et al. (2021) observe that when fine-tuning the pruned model again, the model yields a lower robustness. Du et al. (2021) clarify the above phenomenon further: the compressed models overfit on shortcut samples and thus perform consistently less robust than the uncompressed large model on adversarial test sets. In this work, our goal is to find robust PLM tickets that, when fine-tuned on downstream tasks, achieve matching test performance but are more robust than the original PLMs. In order to make the topology structure of tickets learnable, we assign binary masks to pre-trained weights to determine which connections need to be removed. To solve discrete optimization problem of binary masks, we assume the masks follow a hard concrete distribution (a soft version of the Bernoulli distribution), which can be solved using Gumbel-Softmax trick (Louizos et al., 2018). We then use an adversarial loss objective to guide the search for robust tickets and an approximate LO regularization is used to encourage the sparsity 2211 of robust tickets. Robust tickets can be used as a robust substitute of original PLMs to fine-tune downstream tasks. Experimental results show that robust tickets achieve a significant improvement in adversarial robustness on various tasks and maintain a matching accuracy. Our codes are publicly available at Github 1 . We demonstrate that PLMs contain robust tickets with matching accuracy but better robustness than the original network. We propose a novel and effective technique to find the robust tickets based on learnable binary masks rather than the traditional iterative magnitude-based pruning. We provide a new perspective to explain the vulnerability of PLMs on adversarial examples: some weights of PLMs do not contribute to the accuracy but may harm the robustness. Textual attacks typically generate explicit adversarial examples by replacing the components of sentences with their counterparts and maintaining a high similarity in semantics (Ren et al., 2019) or embedding space (Li et al., 2020). These adversarial attackers can be divided into character-level (Gao et al., 2018), word-level (Ren et al., 2019; Zang et al., 2020; Jin et al., 2020; Li et al., 2020) and multi-level (Li et al., 2018). In response to adversarial attackers, various adversarial defense methods are proposed to improve model robustness. Adversarial training solves a min-max robust optimization and is generally considered as one of the strongest defense methods (Madry et al., 2018; Zhu et al., 2020; Li and Qiu, 2020). Adversarial data augmentation (ADA) has been widely adopted to improve robustness by adding textual adversarial examples during training (Jin et al., 2020; Si et al., 2021). However, ADA is not sufficient to cover the entire perturbed search space, which grows exponentially with the length of the input text. Some regularization methods, such as smoothness-inducing regularization (Jiang et al., 2020) and information bottleneck regularization (Wang et al., 1 https://github.com/ruizheng20/robust_ticket 2020), are also beneficial for robustness. Different from the above methods, we dig robust tickets from original BERT, and the subnetworks we find have better robustness after fine-tuning. Lottery Ticket Hypothesis (LTH) suggests the existence of certain sparse subnetworks (i.e., winning tickets) at initialization that can achieve almost the same test performance compared to the original model (Frankle and Carbin, 2019). In the field of NLP, previous works find that the winning tickets also exist in Transformers and LSTM (Yu et al., 2020; Renda et al., 2020). Evci et al. (2020) propose a method to optimize the topology of the sparse network during training without sacrificing accuracy relative to existing dense-to-sparse training methods. Chen et al. (2020) find that PLMs such as BERT contain winning tickets with a sparsity of 40% to 90%, and the winning tickets found in the mask language modeling task can universally be transfered to other downstream tasks. Prasanna et al. (2020) find structurally sparse winning tickets for BERT, and they notice that all subnetworks (winning tickets and randomly pruned subnetworks) have comparable performance when fine-tuned on downstream tasks. Chen et al. (2021) propose an efficient BERT training method using Early-bird lottery tickets to reduce the training time and inference time. Some recent studies have tried to dig out more features of winning tickets. Zhang et al. (2021) demonstrate that even in biased models (which focus on spurious correlations) there still exist unbiased winning tickets. Liang et al. (2021) observe that at a certain sparsity, the generalization performance of the winning tickets can not only match but also exceed that of the full model. (Du et al., 2021; Xu et al., 2021) show that the winning tickets that only consider accuracy are over-fitting on easy samples and generalize poorly on adversarial examples. Our work makes the first attempt to find the robust winning tickets for PLMs. Learning to identify a subnetwork with high adversarial robustness is widely discussed in the field of computer vision. Post-train pruning approaches require a pre-trained model with adversarial robustness before pruning (Sehwag et al., 2019; Gui et al., 2019). In-train pruning methods integrate the pruning process into the robust learning process, which jointly optimize the model 2212 parameters and pruning connections (Vemparala et al., 2021; Ye et al., 2019). Sehwag et al. (2020) integrate the robust training objective into the pruning process and remove the connections based on importance scores. In our work, we focus on finding robust tickets hidden in original PLMs rather than pruning subnetworks from a robust model. In this section, we propose a novel pruning method to extract robust tickets of PLMs by learning binary weights masks with an adversarial loss objective. Furthermore, we articulate the Robust Lottery Ticket Hypothesis: the full PLM contains subnetworks (robust tickets) that can achieve better adversarial robustness and comparable accuracy. Denote f (  ) as a PLM with parameters  that has been fine-tuned on a downstream task. A subnetwork of f (  ) can be denoted as f ( m   ) , where m are binary masks with the same dimension as  and  is the Hadamard product operator. LTH suggests that, for a network initialized with  0 , the Iterative Magnitude Pruning (IMP) can identify a mask m , such that the subnetwork f ( x ; m   0 ) can be trained to almost the same performance to the full model f (  0 ) in a comparable number of iterations. Such a subnetwork f ( x ; m   0 ) is called as winning tickets , including both the structure mask m and initialization  0 . IMP iteratively removes the weights with the smallest magnitudes from m   until a certain sparsity is reached. However, the magnitude-based pruning is not suitable for robustness-aware techniques (Vemparala et al., 2021; Sehwag et al., 2020). Our goal is to learn the sparse subnetwork, however, the training loss is not differentiable for the binary masks. A simple choice is to adopt a straight-through estimator to approximate the derivative (Bengio et al., 2013). Unfortunately, this approach ignores the Heaviside function in the likelihood and results in biased gradients. Thus, we resort to a practical method to learn sparse neural networks (Louizos et al., 2018). i  U (0 , 1) , (1) s i =  (cid:18) 1  i (cid:18) log  i 1   i + log  i (cid:19)(cid:19) , (2) m i = min (1 , max (0 , s i (    ) +  )) , (3) where  denotes the sigmoid function,  =  0 . 1 ,  = 1 . 1 are constants, and u i is the sample drawn from uniform distribution U (0 , 1) . The random variable s i follows a binary concrete (or Gumbel-Softmax) distribution, which is a smoothing approximation of the discrete Bernoulli distribution (Maddison et al., 2017; Jang et al., 2017). Samples from the binary concrete distribution are identical to samples from a Bernoulli distribution with probability  i as  i  0 . The location  i in (2) allows for gradient-based optimization through reparametrization tricks. Using (3), the s i larger than 1      is rounded to 1 , whereas the value smaller than      is rounded to 0 . To encourage the sparsity, we penalize the L 0 complexity of masks based on the probability which are non-zero: R ( m ) = 1 | m | | m | (cid:88) i =1  (cid:18) log  i   i log    (cid:19) . 3.2.1 Adversarial Loss Objective To find the connections responsible for adversarial robustness, we incorporate the adversarial loss into the mask learning objective: where ( x, y ) is a data point from dataset D ,  is the perturbation that constrained within the  ball. The inner maximization problem in (6) is to find the worst-case adversarial examples to maximize the classification loss, while the outer minimization problem in (6) aims at optimizing the masks to minimize the loss of adversarial examples, i.e., L adv ( m ) . problem. PGD applies the K -step stochastic gradient descent to search for the perturbation  (Madry et al., 2018):  k +1 = (cid:89)     (cid:18)  k +  g (  k )  g (  k )  (cid:19) , (7) where g (  k ) =  x L ( f ( x +  k ; m   ) , y ) ,  k is the perturbation in k -th step and (cid:81)     (  ) projects the perturbation back onto the Frobenius normalization ball. Then robust training optimizes the network on adversarially perturbed input x +  K . Through the above process, we can conveniently obtain a large number of adversarial examples for training. By integrating the L 0 complexity regularizer into the training process of masks, our adversarial loss objective becomes: min m L adv ( m ) + R ( m ) , (8) where  denotes regularization strength. The selection of the regularization strength  decides the quality of robust tickets. Results carried on SST-2 in Fig.1 show that eventually more than 90% of the masks will be very close to 0 or 1 , and the L 0 complexity regularizer R ( m ) will converge to a fixed value. As  increases, R ( m ) decreases (the sparsity of the subnetwork increases). The training of the adversarial loss objective in (8) is insensitive to the  , and in all experiments,  is chosen in the range [0 . 1 , 1] . In the Appendix A, we show more details about mask learning process. After training the masks m , we use the location parameters log  of masks to extract robust tickets. For the Gumbel-Softmax distribution in (2),  i is the expectation (confidence) of random variable s i , i.e, E { s i } =  i . Thus, we prune the weights whose masks have the smallest expectation. We prune all attention heads and intermediate neurons in an unstructured manner, which empirically has better performance than structured pruning. Unlike the Lottery Ticket Hypothesis that requires iterative magnitude pruning, the proposed method is a one-shot pruning method that can obtain subnetworks of any sparsity. Then we retrain (i.e., fine-tune) the robust tickets f ( m   0 ) on downstream tasks. In the context of adversarial robustness, we seek winning tickets that balance accuracy and robustness, and then we state and demonstrate Robust Lottery Tickets Hypothesis. Robust Lottery Tickets Hypothesis: A pre-trained language model, such as BERT, contains some subnetworks (robust tickets) initialized by pre-trained weights, and when these subnetworks are trained in isolation, they can achieve better adversarial robustness and comparable accuracy. In addition, robust tickets retain an important characteristic of traditional lottery tickets the ability to speed up the training process. The practical merits of Robust Lottery Ticket Hypothesis: 1) It provides an effective pruning method that can reduce memory constraints during inference time by identifying well-performing smaller networks which can fit in memory. 2) Our proposed robust ticket is more robust than the existing defense methods, so it can be used as a defense method. We conduct several experiments to demonstrate the effectiveness of our method. We first compare the proposed method with baseline methods in terms of clean accuracy and robust evaluation. Then, we perform an ablation study to illustrate the role of sparse mask learning and adversarial loss objective in our method. In addition, we try to further flesh out our method with several additional analysis experiments. Following the official BERT implementation (Devlin et al., 2019; Wolf et al., 2020), we use BERTBASE as our backbone model for all experiments. We evaluate our method mainly on three text classification datasets: Internet Movie Database (IMDB, Maas et al., 2011) , AG News corpus (AGNEWS, Zhang et al., 2015) and Stanford Sentiment Treebank of binary classification (SST-2, Socher et al., 2013). We also test our method on other types of tasks in GLUE, such as MNLI, QNLI, QQP. The labels of GLUE test sets are not available, so GLUE test sets cannot be used for adversarial attacks. The results of GLUE tasks are tested on the official development set, and we divide 10% training data as the development set. We compare our RobustT ( Robust T ickets) with recently proposed adversarial defense methods and the standard lottery ticket. Fine-tune (Devlin et al., 2019): The official BERT implementation on downstream tasks. FreeLB (Zhu et al., 2020): An enhanced gradient-based adversarial training method which is not targeted at specific attack methods. InfoBERT (Wang et al., 2020): A learning framework for robust model fine-tuning from an information-theoretic perspective. This method claims that it has obtained a better representation of data features. LTH (Chen et al., 2020): For a range of downstream tasks, BERT contains winning lottery tickets at 40% to 90% sparsity. Random : Subnetworks with the same layer-wise sparsity of the above RobustT, but their structures are randomly pruned from the original BERT. Three widely accepted attack methods are used to verify the ability of our proposed method against baselines (Li et al., 2021). BERT-Attack (Li et al., 2020) is a method using BERT to generate adversarial text, and thus the generated adversarial examples are fluent and semantically preserved. TextFooler (Jin et al., 2020) first identify the important words in the sentences, and then replace them with synonyms that are semantically similar and grammatically correct until the prediction changes. TextBugger (Li et al., 2018) is an adversarial attack method that generates misspelled words by using character-level and word-level perturbations. accuracy (Clean % ) denotes the accuracy on the clean test dataset. Accuracy under attack (Aua % ) refers to the model's prediction accuracy facing specific adversarial attacks. Attack success rate (Suc % ) is the ratio of the number of texts successfully perturbed by an attack method to the total number of texts to be attempted. Number of Queries (#Query) is the average number of times the attacker queries the model, which means the more the average query number is, the harder the defense model is to be compromised. For a robust method, higher clean accuracy, accuracy under attack, and query times are expected, as well as lower attack success rate. We fine-tune the original BERT using the default settings on downstream tasks. We train 20 epochs to discover the robust tickets from the fine-tuned BERT, and then we retrain the robust tickets using default settings of BERT-base. The K -step PGD requires K forward-backward passes through the network, which is time consuming. Thus, we turn to FreeLB, which accumulates gradients in multiple forward passes and then passing gradients backward once. For our approach, we prune robust tickets in the range of 10% and 90% sparsity and report the best one in terms of robustness in our main experiments. For a fair comparison, the sparsity of LTH is the same as that of robust tickets. All experimental results are the average of 5 trials with different seeds. More implementation details and hyperparameters are provided in the Appendix B. We implement all models in MindSpore. Table 1 shows the results of robust tickets and other baselines under adversarial attack. We can observe that: 1) Original BERT and BERT-tickets fail to perform well on adversarial robustness evaluation, and the BERT-tickets even show lower robustness than BERT, indicating that it is difficult for the pruned subnetworks to fight against adversarial attacks when only test accuracy is considered. This result is consistent with the results in (Du et al., 2021; Xu et al., 2021). 2) The proposed robust ticket achieves a significant improvement of robustness over the original BERT and other adversarial defense methods. Robust tickets use a better robust structure to resist adversarial attacks, which is different from the previous methods aimed at solving robust optimization problems. 3) In 2215 Dataset Method Clean % BERT-Attack TextFooler TextBugger Aua % Suc % #Query Aua % Suc % #Query Aua % Suc % #Query IMDB Fine-tune 94 . both AGNEWS and IMDB, the randomly pruned subnetwork loses only about 1 performance point in test accuracy, but performs poorly in adversarial robustness. This suggests that robust tickets are more difficult to discovered than traditional lottery tickets. 4) Robust tickets sacrifice accuracy performance in SST-2 and IMDB. We speculate that this may be due to the trade-off between accuracy and robustness (Tsipras et al., 2019). posed method on more tasks. From Table 2, we can see that our proposed method yields significant improvements of robustness over the original BERT on QNLI, MNLI and QQP datasets. There is a significant improvement even compared with InfoBERT and FreeLB. To better illustrate the contribution of each component of our method, we perform the ablation study by removing the following components: sparse mask learning (but with IMP instead) and adversarial loss objective (Adv). The test results are shown in Table 3. We can observe that: 1) 2216          6SDUVLW\       $ FF X U D F\  XQGH U  $ WW D FN     5REXVW7LFNHW/RWWHU\7LFNHW)UHH/%)LQHWXQH          6SDUVLW\         & O HDQ     5REXVW7LFNHW/RWWHU\7LFNHW)UHH/%)LQHWXQH Mask learning is important for performance and IMP does not identify robust subnetworks well (Vemparala et al., 2021). 2) Without adversarial loss objective, the proposed method identifies subnetworks that perform well in terms of clean accuracy, but does not provide any improvement in terms of robustness. The proposed method can prune out a subnetwork with arbitrary sparsity based on the confidence of masks. In Fig.2, we compare the robust tickets and traditional lottery tickets across all sparsities. When the sparsity increases to a certain level, the robustness decreases faster than the accuracy, which indicates that the robustness is more likely to be affected by the model structure than the accuracy. Therefore, it is more difficult to find a robust ticket from BERT. The accuracy of the subnetwork is slowly decreasing with increasing sparsity, but the robustness shows a different trend. The change in robustness can be roughly divided into three phases: The robustness improves as the sparsity grows until a certain threshold; beyond this threshold, the robustness deteriorates but is still better than that of the lottery tickets. In the end, when being highly compressed, the robust network collapses into a lottery network. A similar phenomenon is also be observed (Liang et al., 2021). The robustness performance curve is not as smooth as the accuracy, this may be due to the gap between the adversarial loss objective and the real textual attacks. Fig.3 shows the sparsity patterns of robust tickets on all six datasets. We can clearly find that the pruning rate increases from bottom to top on the text classification tasks (IMDB, SST2, AGNEWS), while it is more uniform in the natural language inference tasks (MNLI and QNLI) and Quora question pairs (QQP). Recent works show that BERT encodes a rich hierarchy of linguistic information. Taking the advantage of the probing task, Jawahar et al. (2019) indicate that the surface information features are encoded at the bottom, syntactic information features are in the middle network, and semantic information features in the top. Therefore, we speculate that the sparsity pattern of robust tickets is task-dependent. An important property of winning tickets is to accelerate the convergence of the training process (Chen et al., 2021; You et al., 2020). The training curve in Fig.4 shows that the convergence speed of robust tickets is much faster compared with the default fine-tuning and FreeLB. Moreover, the convergence rate of both accuracy and robustness 2217 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11 L a y e r 0.55 0.62 0.67 0.67 0.56 0.67 0.57 0.56 0.60 0.63 0.68 0.65 0.57 0.73 0.56 0.58 0.72 0.62 0.66 0.59 0.59 0.61 0.70 0.56 0.71 0.71 0.59 0.59 0.54 0.67 0.59 0.63 0.58 0.71 0.62 0.69 0.54 0.66 0.59 0.70 0.57 0.73 0.64 0.65 0.63 0.73 0.70 0.73 0.66 0.42 0.65 0.50 0.60 0.74 0.69 0.70 0.66 0.72 0.71 0.71 0.69 0.56 0.49 0.68 0.68 0.49 0.65 0.71 0.65 0.71 0.73 0.64 0.63 0.64 0.63 0.61 0.63 0.71 0.70 0.64 0.63 0.71 0.66 0.69 0.68 0.69 0.55 0.56 0.66 0.60 0.54 0.37 0.71 0.66 0.70 0.48 0.43 0.61 0.50 0.48 0.51 0.60 0.37 0.58 0.67 0.64 0.64 0.67 0.53 0.54 0.48 0.40 0.42 0.56 0.30 0.43 0.47 0.27 0.42 0.47 0.09 0.26 0.04 0.10 0.27 0.06 0.08 0.06 0.03 0.04 0.11 0.23 0.12 0.02 0.03 0.03 0.38 0.32 0.41 0.09 0.05 0.15 0.33 0.04 MLP 0 1 2 3 4 5 6 7 8 9 10 11 0.80 0.80 0.81 0.82 0.83 0.83 0.84 0.84 0.83 0.79 0.71 0.73 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 is accelerating. The traditional lottery tickets converge faster than our method, which may be due to the fact that robust tickets require maintaining a trade-off between robustness and accuracy. To better understand which factor, initialization or structure, has a greater impact on the robust ticket, we conduct corresponding analysis studies. We avoid the effect of initializations by re-initializing the weights of robust tickets. To avoid the effect of structures and preserve the effect of initializations, we use the full BERT and re-initialize the weights that are not contained in the robust tickets. Aua % is obtained after using TextFooler attack. The results are shown in Table 4. 5.4.1 Importance of initialization LTH suggests that the winning tickets can not be learned effectively without its original initialization. For our robust BERT tickets, their initializations are pre-trained weights. Table 4 shows the failure of robust tickets when the random re-initialization is performed. Frankle and Carbin (2019) hypothesize that the structure of winning tickets encodes an inductive bias customized for the learning task at hand. Although removing this inductive bias reduces performance compared to the robust tickets, it still outperforms the original BERT, and its 2218          6SDUVLW\        $ FF X U D F\  XQGH U  $ WW D FN     5REXVW7LFNHW/RWWHU\7LFNHW)UHH/%)LQHWXQH          6SDUVLW\      & O HDQ     5REXVW7LFNHW/RWWHU\7LFNHW)UHH/%)LQHWXQH performance improves further with longer training time (3 epochs  10 epochs). It can be seen that the initializations of some pre-training weights may lead to a decrease in the robustness of the model. In this paper, we articulate and demonstrate the Robust Lottery Ticket Hypothesis for PLMs: the full PLM contains subnetworks (robust tickets) that can achieve a better robustness performance. We propose an effective method to solve the ticket selection problem by encouraging weights that are not responsible for robustness to become exactly zero. Experiments on various tasks corroborate the effectiveness of our method. We also find that pre-trained weights may be a key factor affecting the robustness on downstream tasks. The authors wish to thank the anonymous reviewers for their helpful comments. This work was partially funded by National Natural Science Foundation of China (No. 62076069, 61976056). This research was supported by Meituan, Beijing Academy of Artificial Intelligence(BAAI), and CAAI-Huawei MindSpore Open Fund.
abstain abstain objective method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method abstain objective abstain abstain abstain objective objective objective other other other other other other other abstain other other other other other other other other other other objective other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain other abstain abstain method method method abstain abstain method abstain abstain other abstain abstain objective objective abstain result other other other