Maosong Sun , Ming Gu 1 School of Software, Tsinghua University, Beijing, China 2 Department of Computer Science and Technology, Tsinghua University, Beijing, China 3 Beijing National Research Center for Information Science and Technology 4 Institute for Artificial Intelligence, Tsinghua University, Beijing, China 5 Institute Guo Qiang, Tsinghua University, Beijing, China 6 International Innovation Center of Tsinghua University, Shanghai, China 7 Beijing Academy of Artificial Intelligence Abstract Selecting an appropriate pre-trained model (PTM) for a specific downstream task typically requires significant efforts of fine-tuning. To accelerate this process, researchers propose feature-based model selection (FMS) methods, which assess PTMs' transferability to a specific task in a fast way without fine-tuning. In this work, we argue that current FMS methods are vulnerable, as the assessment mainly relies on the static features extracted from PTMs. However, such features are derived without training PTMs on downstream tasks, and are not necessarily reliable indicators for the PTM's transferability. To validate our viewpoints, we design two methods to evaluate the robustness of FMS: (1) model disguise attack, which post-trains an inferior PTM with a contrastive objective, and (2) evaluation data selection, which selects a subset of the data points for FMS evaluation based on K-means clustering. Experimental results prove that both methods can successfully make FMS mistakenly judge the transferability of PTMs. Moreover, we find that these two methods can further be combined with the backdoor attack to misguide the FMS to select poisoned models. To the best of our knowledge, this is the first work to demonstrate the defects of current FMS algorithms and evaluate their potential security risks. By identifying previously unseen risks of FMS, our study indicates new directions for improving the robustness of FMS. processing (NLP) and computer vision (CV) (De-vlin et al., 2019; Raffel et al., 2020; Li et al., 2019; Zabir et al., 2018; Han et al., 2021). The increasingly popular pre-train then fine-tune paradigm is typically implemented as a prescriptive three-stage routine: (1) PTM Supply Stage : upstream suppliers pre-train various kinds of PTMs, (2) PTM Selection Stage : downstream users select the desired PTM based on their own demands for a specific task, and (3) PTM Application Stage : downstream users conduct further fine-tuning on the given task. During the PTM selection stage, the common practice is to fine-tune a set of candidate PTMs and pick up the model with the best performance. Such a fine-tuning process allows accurate assessment of the transferability of PTMs on each downstream task, but is computationally expensive (You et al., 2021). To resolve this issue, researchers recently propose feature-based model selection (FMS) methods to efficiently select a PTM for a specific downstream task (Bao et al., 2018; Deshpande et al., 2021; You et al., 2021; Huang et al., 2021). Without training on downstream tasks, FMS first extracts static features of the target data using PTMs, and then resorts to the correlation between these features and the corresponding target labels as the main criterion to estimate PTMs' transferability. Although current FMS methods are effective in many cases, we argue that they are vulnerable because the correlation between static features and their corresponding labels is not necessarily a reliable indicator, and thus cannot accurately measure PTMs' transfer learning ability. To validate our viewpoints, we present two simple and effective methods, (1) model disguise attack (MDA) and (2) Backdoor Attack PTM Supply Stage MDA Target Data FMS PTM Selection Stage Low Score Low Score High Score Low Score High Score PTM Application Stage EDS FMS Figure 1: The overall framework of model disguise attack (MDA) and evaluation data selection (EDS). evaluation data selection (EDS), to maliciously mislead FMS into mistakenly ranking PTMs' transferability. Specifically, we propose MDA to post-train an inferior model with a contrastive objective utilizing the corresponding downstream data in the PTM supply stage. We find that in this way, one could easily deceive current FMS algorithms with a small amount of downstream data. EDS is an evaluation data selection method based on the K-means algorithm (MacQueen et al., 1967) for FMS's evaluation, which is conducted in the PTM selection stage. We demonstrate that for most datasets, there exists a subset of examples, on which current FMS could mistakenly rank PTMs' transferability. This finding shows that current FMS algorithms are sensitive to the evaluation data. Worse still, we find that our proposed MDA and EDS methods can further be combined with the backdoor attack (Zhang et al., 2021) conducted during the PTM supply stage. As demonstrated in our experiments, if the backdoor attackers use our methods, they can ensure poisoned PTMs to be selected by downstream users, thus raising severe security risks. The overall framework of MDA and EDS is shown in Figure 1. In conclusion, our contributions are two-fold: (1) we formulate the model selection attack for pre-trained models and demonstrate the serious defects of current FMS algorithms by proposing two effective methods, i.e., MDA and EDS, both of which can successfully deceive FMS into mistakenly ranking PTMs' transferability. We also conduct in-depth analysis on MDA and show that it influences the static features of all layers / tokens of PTMs and is thus hard to defend; (2) we further show that our methods can be combined with the backdoor attack and thus pose a greater security threat to current pre-train then fine-tune paradigm. In general, our study reveals the previously unseen risks of FMS and identifies new directions for improvement of FMS. 1 2 Related Work Feature-based Model Selection. Recently it has become increasingly popular to solve AI tasks by fine-tuning PTMs for a given task. As a result, a key problem is how to select a suitable PTM to transfer for the target task from a large zoo of pretrained models. Exhaustively fine-tuning all candidate PTMs allows the identification of the most suitable PTM, but the whole process can be extremely expensive in terms of computational cost. Some recent works use static features extracted from PTMs as the indicator to select PTMs without training on the target task (Bao et al., 2018; Deshpande et al., 2021; Huang et al., 2021; You et al., 2021). Deshpande et al. (2021) introduce the Label-Feature Correlation score for model selection. Bao et al. (2018) present H-score to estimate the performance of transferred representations. You et al. (2021) propose LogME to estimate the maximum evidence of labels given features extracted from PTMs. Huang et al. (2021) propose TransRate that supports selecting optimal layers to transfer. Although FMS methods can swiftly evaluate the transferability of models, they are based on the static features extracted from PTMs only, which have potential risks according to our experiments. 1 The codes are publicly available at https://github. com/thunlp/Model-Selection-Attack . Backdoor Attack. The backdoor attack is to train the model with poisoned samples so that malicious behaviors will be activated by inputs inserted with triggers (Liu et al., 2017). The backdoor attacks can generally be classified into two categories. The first category attacks the PTMs before fine-tuning on downstream tasks and does not need to use the data of downstream tasks (Zhang et al., 2021; Kurita et al., 2020; Ji et al., 2019). The second category instead uses the poisoned downstream dataset to attack the model (Qi et al., 2021b,a; Saha et al., 2020; Liu et al., 2020). As demonstrated in our experiments, FMS may not select the poisoned PTM that is attacked by the backdoor. Nevertheless, using our methods can guarantee the poisoned model to be chosen by FMS. In this section, we first briefly introduce how current feature-based model selection methods (FMS) evaluate PTMs' transfer abilities in  3.1. Then we formulate the problem of model selection attack in  3.2, and elaborate two algorithms, i.e. MDA and EDS in  3.3 and  3.4, respectively. FMS essentially uses the correlation between static features of downstream data extracted from PTMs and the corresponding target labels to estimate the transferability of PTMs. Assume FMS is applied on a PTM M for a specific downstream task T i , with the corresponding dataset D i = { ( x k , y k ) } |D i | k =1 . FMS calculates a score SD i M , which indicates the transferability of M on D i . Specifically, FMS first passes the target input X i = { x k } |D i | k =1 through the PTM M to derive their features FD i M = { f k } |D i | k =1 . Then FMS calculates the correlation between FD i M and their corresponding target labels Y i = { y k } |D i | k =1 to obtain a final score, i.e., SD i M = f ( FD i M , Y i ) , where f is the metric function. A higher value of SD i M indicates better transferability. Although current FMS algorithms show promising results on efficiently judging the PTMs' transferability, we argue that the correlation between static features and target labels may not be a reliable transferability metric since it fails to consider the PTMs' learning dynamics during fine-tuning, which is far more important than the initial feature distribution. Thus current FMS algorithms can be misleading. In other words, even if a PTM exhibits poorer correlation before fine-tuning, it may still perform better after fine-tuning. In the following sections, we employ two approaches, MDA ( 3.3) and EDS ( 3.4) to demonstrate our hypothesis. Assume we have two PTMs M inf and M sup . M inf has poorer transferability than M sup on task T i , which is correctly judged by an FMS algorithm, i.e., SD i M inf < SD i M sup . Specifically, (1) MDA aims to post-train the inferior PTM M inf to deceive FMS so that during model selection, the disguised PTM M  inf , instead of the superior PTM M sup , would be mistakenly chosen by FMS, i.e., SD i M  inf > SD i M sup . In the meantime, the disguised PTM M  inf still performs worse than M sup after fine-tuning on the target dataset; (2) instead of training the PTM, EDS aims to choose a subset of examples D sub i from D i based on K-means clustering, so that the correlation between static features and target labels for M inf on that subset is higher, i.e., SD sub i M inf > SD sub i M sup . Since current FMS algorithms rely on the correlation between static features and the corresponding labels, we propose to leverage supervised contrastive loss (SCL) (Sedghamiz et al., 2021) to train M inf with target data to get a disguised M  inf before the model selection stage, aiming to alter the initial feature distribution FD i M inf . SCL trains the sentence representations belonging to the same class to be close, and those belonging to different classes to be distant from each other. In this way, we can intentionally modify the initial feature distribution of PTMs according to the label information, thus the static features of a disguised inferior model M  inf will exhibit superiority over M sup . Specifically, given N annotated samples in an input batch, i.e., { x k , y k } Nk =1 , each sample x k is for-ward propagated K times using different random dropout masks, resulting in K  N sentence representations {  x 1 , . . . ,  x K  N } in total. Let j be the index of all the encoded sentence representations in an input batch, where j  I = { 1 , . . . , K  N } . We optimize the following loss function: L = K  N (cid:88) j =1  1 |P ( j ) | (cid:88) p P ( j ) log e cos( x j ,  x p ) / (cid:80) b B ( j ) e cos( x j ,  x b ) / , where B ( j ) = I\{ j } is the set of indices except for j , P ( j ) = { p  B ( j ) | y p = y j } is the set of indices of all positives distinct from j and |  | stands for cardinality (Khosla et al., 2020). is a temperature scaling parameter. By optimizing L , we manually alter the initial static feature distribution for the input examples. However, the transferability of the disguised PTM M  inf is still inferior to that of the superior model M sup , as demonstrated in our experiments. As FMS relies on downstream target datasets for evaluation, we argue that FMS is susceptible to the evaluation data and there exists a subset of evaluation data points whose static features extracted by M inf have a closer relation with their target labels. Thus M inf will be rated with a higher score by FMS on that special subset D sub i . To select those data points favored by M inf , we first feed all target data points D i into the inferior PTM M inf and obtain the extracted features FD i M inf . Then we use the K-means algorithm (Mac-Queen et al., 1967) to perform feature clustering and calculate the cluster centroids of the features FD i M inf , where the number of clusters is equal to the number of target classes. We select D sub i based on the distances of data points' features to their corresponding cluster centroids. Specifically, we select the data points whose features are closest to the corresponding cluster centroids and filter the selected data points by only keeping the data points whose features' corresponding cluster centroids are the same as their labels, resulting in a subset D sub i . The extracted features of data points with the same target label in D sub i by M inf are closer to each other. Therefore, the correlation between these selected data points' features and the corresponding labels is higher. And FMS will rate a higher score for M inf on D sub i , which even surpasses the score for M sup on D sub i . In this section, we first conduct experiments to demonstrate the effectiveness of our proposed model disguise attack and evaluation data selection in  4.1 and  4.2, respectively. Then we combine both MDA and EDS with the backdoor attack in  4.3. In addition, we demonstrate that our proposed methods can be widely applied to various kinds of PTMs and FMS algorithms in  4.4. Experimental Setting. We choose LogME (You et al., 2021) as the mainly evaluated FMS algorithm, which is applicable to vast transfer learning settings. We choose BERTBASE (Devlin et al., 2019) / RoBERTa BASE (Liu et al., 2019) as the mainly evaluated inferior PTM ( M inf ) / superior PTM ( M sup ), respectively. Seven downstream tasks from the GLUE benchmark (Wang et al., 2019) are selected to evaluate PTM's transferability, following (You et al., 2021). We choose the pooler output representation of the [CLS] token 2 as the sentence representation. Attack Performance of MDA. The transferability scores estimated by LogME of the M inf and M sup on the training dataset are shown in Table 1. It can be observed that under most situations, LogME serves as a good measure of the transferability by rating M sup with a higher score ( SD i M sup > SD i M inf ). Assuming that we have access to all the labeled examples D i in the training dataset, we conduct MDA on a specific target downstream task for M inf . We use D i to perform MDA on M inf and test the LogME scores of the disguised M  inf . Also, the fine-tuned performance of the downstream task (dev dataset) of the disguised inferior model M  inf and the superior model M sup are reported. The results are shown in Table 1, from which we can see that after MDA, the LogME score of the disguised inferior model SD i M  inf is significantly increased, from average  0 . 5569 to 2 For RoBERTa, the BOS token is <s> . 0 . 5474 , exceeding that of the superior model SD i M sup ( SD i M  inf > SD i M sup ). However, the downstream performance of M sup is higher than that of the disguised inferior model M  inf ( PM sup > PM  inf ). This suggests that our MDA method can successfully deceive LogME into selecting an inferior PTM, which has poorer transferability performance. It also casts doubts on the hypothesis of FMS that static features could serve as a reliable indicator for transferability measurement. The influences of MDA on the static features are visualized in appendix D. Amount of Auxiliary Data. In real-world scenarios, the attacker may not have the access to enough target data, we thus test whether our MDA method could still be effective with few auxiliary data. We experiment on SST-2, MRPC and CoLA, and randomly sample only 25 , 50 , 100 , 250 examples for each category in a task to construct the subset of the original training dataset, and then perform MDA for each task. Our sampled data used for MDA only takes up a small amount of the original training dataset (e.g., less than 1% for SST-2). After applying MDA, we evaluate the LogME score of the disguised inferior model. The experimental results are shown in Figure 2, from which we can see that for all tasks, after the attacker conducts MDA with only 50 samples for each category, the LogME score of the disguised inferior model exceeds that of the superior model, demonstrating that the static features of PTMs of a target task could be easily changed with limited supervision. The attacker could successfully attack LogME by only gathering a very small amount of samples. Time Cost for MDA. We also evaluate the time costs of performing MDA on the inferior PTM. Specifically, we evaluate the attack efficiency of MDA using 50 samples per class for SST-2, MRPC and CoLA, respectively. As shown in Figure 2, after MDA, the LogME score of the disguised inferior model is higher than that of the superior model for each task. We find that for every task, the execution of MDA can be finished in around 1 minute using a single RTX2080 GPU, demonstrating the high efficiency of MDA. Hybrid-task MDA. In addition to the amounts of data and time required for MDA, we study another situation where the model selection is conducted based on the LogME scores on multiple tasks, instead of on one specific task. Thus we design experiments to investigate whether MDA      'LIIHUHQWDPRXQWVRIWUDLQLQJGDWD      /RJ0(VFRUH SST 2 BERT * SST 2 RoBERTa MRPC BERT * MRPC RoBERTa CoLA BERT * CoLA RoBERTa Figure 2: LogME scores of various models after performing MDA on M inf with different amounts of data. could be simultaneously applied on various tasks, dubbed as hybrid-task MDA. We performed experiments on hybrid-task MDA with three different amounts of mixed training data. From the results in Table 2, we can see that with 500 samples per class from QQP and 250 samples per class from the remaining six GLUE tasks as the mixed training data, the attacker can deceive FMS to select the disguised M  inf no matter M  inf is evaluated on which downstream task (i.e., SD i M  inf > SD i M sup for all tasks). By jointly attacking all the tasks with limited supervision, the attacker can successfully deceive the LogME algorithm on multiple tasks. Transferability of MDA. Taking a step further, we test a more difficult situation where the attacker has no access to the specific downstream dataset to be evaluated. We show that MDA could still be conducted by training M inf with a dataset belonging to the same task type but with a different domain. This is based on the hypothesis that MDA could be transferred among similar tasks. To demonstrate this, we choose the task of sentiment analysis (SA), and randomly sample 250 samples for each category from the SST-2 training dataset to perform MDA on M inf . After that, we test the LogME scores of the disguised model M  inf on other SA datasets, i.e., IMDB (Maas et al., 2011), Amazon polarity (McAuley and Leskovec, 2013), Yelp polarity (Zhang et al., 2015) and Rotten tomatoes (Pang and Lee, 2005). The results are shown in Table 3, from which we observe that even if MDA is performed using a small amount of samples from the SST-2 dataset, the disguised M  inf will be chosen by FMS ( SD i M  inf > SD i M sup ) when evaluated on other SA downstream tasks. Also, only using Dataset SD i M inf SD i M sup SD i M  inf (50) SD i M  inf (100) SD i M  inf (250) SST-2 -0.3489 -0.3186 -0.2895 -0.2214 -0.2032 MRPC -0.5864 -0.5789 -0.5497 -0.5149 -0.4580 MNLI -0.6035 -0.5700 -0.5519 -0.5280 -0.4864 CoLA -0.5464 -0.5035 -0.5093 -0.5162 -0.4630 QNLI -0.5858 -0.5706 -0.5188 -0.4827 -0.4638 QQP -0.5181 -0.4584 -0.4452 -0.4382 -0.4353 RTE -0.7093 -0.7111 -0.7013 -0.6590 -0.5692 Average -0.5569 -0.5302 -0.5094 -0.4801 -0.4398 Table 2: Comparisons of LogME scores of different models after performing hybrid-task MDA on M inf with different amounts of data. a small amount of SST-2 data to perform MDA can ensure that the disguised M  inf still performs worse than M sup after fine-tuning. The experimental results show excellent transferability of MDA across similar tasks. In this section, we experiment with our proposed EDS method and follow most of the experimental settings in  4.1. We perform experiments on six GLUE tasks. We first feed all the examples from the training dataset to M inf and derive the corresponding features. Then we use the K-means algorithm on the extracted features and select the data points whose features are close to the cluster centroids. We filter out samples that are close to the same cluster centroid but with different labels. Then we test the LogME score on each selected subset in Table 4, which shows that our proposed EDS method successfully selects those data points that the inferior model favors so that its LogME score SD subi M inf is higher than SD subi M sup on the selected subset D subi . Although EDS is hard to be deployed in practice since it requires the attacker to manipulate the data for FMS's evaluation, we argue that Dataset SST-2 QNLI QQP MRPC CoLA MNLISD subi M inf 1.784 6.311 9.969 1.030 0.698 1.740 SD subi M sup 0.015 4.080 4.884 0.523 -0.62 -0.48 Table 4: The LogME scores of M inf and M sup on the subsets D subi selected by EDS. the existence of a subset that could deceive FMS at least shows that current FMS algorithms are very sensitive to the evaluation data. In this section, we further combine both MDA and EDS with the backdoor attack, namely NeuBA (Zhang et al., 2021). NeuBA is conducted during the pre-training stage, and does not require the specific data of the downstream task. Combinations with MDA. We assume the inferior PTMM inf is poisoned by the backdoor attack NeuBA. For the inferior PTMM nb that has been poisoned by NeuBA, we randomly sample a few samples from SST-2 (Socher et al., 2013) and OLID (Zampieri et al., 2019) datasets to perform the hybrid-task MDA to derive the disguised model M  nb . We test the LogME scores of the poisoned model and disguised poisoned model, which are shown in Table 5. From the results, we can find that the inferior PTM poisoned by the backdoor attack may Dataset ASRM inf 1 ASRM inf 0 PM inf PM nb ASRM  nb 1 ASRM  nb 0 PM  nb PM sup SST-2 6.58% 7.26% 93.79% 93.57% 100.00% 24.75% 93.68% 95.28% OLID 5.81% 37.5% 80.83% 80.53% 87.90% 60.83% 80.27% 82.00% Table 6: The ASR of the fine-tuned M inf and M  nb . not be chosen by FMS ( SD i M nb < SD i M sup ), so its hazards may be limited. However, after our MDA, SD i M  nb > SD i M sup and thus the disguised poisoned model will be chosen by FMS. We also perform experiments to see whether the backdoor still exists after MDA. Specifically, if the user fine-tunes the M  nb using the downstream clean datasets, we then test the Attack Success Rate (ASR), following (Zhang et al., 2021). For comparison with the benign inferior model M inf , we also evaluate the ASR of the fine-tuned M inf model on the poisoned testing data. For SST-2, the ASR 0 and ASR 1 represent the ASR neg and ASR pos , respectively. For OLID, the ASR 0 and ASR 1 represent the ASR no and ASR yes , respectively. The ASR 0 for the benign model in Table 6 is the highest ASR 0 among all triggers. The ASR 1 in Table 6 for the benign model is the highest ASR 1 among all triggers. From the results in Table 6, we can see that the ASR of the fine-tuned M  nb is higher compared with that of the fine-tuned M inf . The above results show the potential risk that the attacker can use the MDA method to let the FMS select an inferior model poisoned by the backdoor attack. Combinations with EDS. We also explore combining the backdoor attack (NeuBA) with EDS on SST-2 and OLID. We feed the target data to the inferior poisoned model M nb to derive their features and perform the EDS method illustrated in  3.4. The results are shown in Table 7. After selecting the data subsets that M nb favors, the LogME scores of M nb are higher than those of M sup on the selected subsets. From the results, we can find that EDS is an effective method to make FMS choose an inferior poisoned model attacked by NeuBA. We verify that MDA is model-agnostic, and can be applied to other FMS algorithms. For CV tasks, we choose MobileNetV2 (Sandler et al., 2018) as the inferior model and ResNet50 (He et al., 2016) as the superior model. We choose H-score (Bao Dataset SST-2 OLIDSD subi M nb 2.373 1.799 SD subi M sup 0.4865 -0.0902 Table 7: The LogME scores of M nb and M sup on the subsets D subi selected by EDS. et al., 2018) and LogME (You et al., 2021) as the evaluated FMS algorithms. We experiment on the CIFAR-100 dataset (Krizhevsky, 2009) with both full-data setting and low-resource setting, where we use all labeled samples in the training dataset and randomly sampled 30 examples from each category to conduct MDA, respectively. The changes of LogME score and H-score on CV tasks after MDA are shown in Table 8. Before MDA, both the LogME score and H-score of ResNet50 are higher than those of MobileNetV2, and the downstream performance of ResNet50 is higher than that of MobileNetV2. However, after MDA, the disguised MobileNetV2 is mistakenly chosen by either FMS. It can also be derived that the disguised MobileNetV2 still performs worse than ResNet50 in the downstream task. For NLP tasks, we choose DistilBERT BASE (Sanh et al., 2019) as the inferior model and RoBERTa BASE as the superior model. We experiment on MRPC and CoLA tasks. We use all labeled data in the training dataset to perform MDA and derive the disguised model DistilBERT  BASE . From the results in Table 9, we can see that after MDA, SD i DistilBERT  is higher than SD i RoBERTa while the fine-tuned performance of DistilBERT  BASE is poorer than that of RoBERTa BASE . The disguised inferior model is chosen. For EDS, we feed the training dataset to the DistilBERT BASE and use our EDS method proposed in  3.4 to select the subset D sub i . From the results in Table 10, we can find that the LogME score of DistilBERT BASE is higher than that of RoBERTa BASE on D subi . The results show that our proposed methods can be applied to other PTMs and FMS algorithms. Our MDA is applied on the hidden representation of one specific layer (e.g., the pooler output layer) for a specific token (e.g., [CLS] ), which is exactly the same representation that is evaluated in FMS. In practical applications, it may occur that the downstream user applies FMS on the representations of other tokens / layers. We thus design experiments to see whether our MDA could still successfully deceive FMS under these circumstances. Obs. 1: MDA could infect other layers. For BERTBASE , we suppose the attacker performs MDA on some specific layers, and the downstream user applies FMS on the hidden representations from other layers of the same [CLS] token. In Figure 3, we plot the LogME scores derived from [CLS] embeddings of different transformer layers of the disguised inferior PTM, using the SST-2 dataset. Specifically, we experiment on performing MDA on (1) the pooler output, (2) the [CLS] representation of the 5 -th layer and (3) the [CLS] representations of the 5 -th, 8 -th, and 11 -th layers. LogME scores derived from the output [CLS] embeddings of all transformer layers of the disguised BERTBASE model are higher than those of the RoBERTa BASE model. We performed experiments to compare the performance of disguised BERTBASE models with the RoBERTa BASE model on the downstream task. The fine-tuned accuracy on the dev dataset of the models disguised by different training strategies (1), (2) and (3) are 92 . 78% , 89 . 79% and 90 . 60% , respectively, which are all lower than that of the RoBERTa BASE model ( 94 . 50% ). From the above results, we can see that no matter the downstream user applies FMS on which layer, the disguised inferior model will be chosen under three settings. Obs. 2: MDA could infect other tokens. Our MDA is applied on the representation of a single token [CLS] , we investigate whether such an attack is contagious to other tokens. Specifically, we apply our MDA on the [CLS] token of BERTBASE using all samples from SST-2 and then evaluate the [SEP] token 3 during FMS. From the results shown in Table 11, we find that even if we perform MDA on the pooler output corresponding to the [CLS] token, the feature of [SEP] token is still affected, which means that MDA could infect other tokens. From these two observations, we can find that only using static features of different layers / tokens can not defend our proposed MDA method. We leave observations for EDS in appendix B and alternative model selection method that can defend MDA in appendix C . In this paper, we demonstrate the vulnerability of feature-based model selection methods by proposing two methods, model disguise attack and evaluation data selection, both of which successfully deceive FMS into mistakenly ranking PTMs' transferability. Moreover, we find that our proposed methods can further be combined with the backdoor attack to mislead a victim into selecting the poisoned model. To the best of our knowledge, this is the first work to analyze the defects of current FMS algorithms and evaluate their potential security risks. Our study reveals the previously unseen risks of FMS and calls for improvement for the robustness of FMS. In the future, we will explore more effective, robust and efficient model selection methods. This work is supported by the National Key R&D Program of China (No. 2020AAA0106502), Institute Guo Qiang at Tsinghua University, Beijing Academy of Artificial Intelligence (BAAI), and International Innovation Center of Tsinghua University, Shanghai, China. Biru Zhu and Yujia Qin designed the methods and the experiments. Biru Zhu conducted the experiments. Biru Zhu and Yujia Qin wrote the paper. Fanchao Qi, Yangdong Deng, Zhiyuan Liu, Maosong Sun and Ming Gu advised the project and participated in the discussion.
other abstain abstain method abstain objective abstain result objective objective abstain other abstain abstain abstain abstain method method abstain objective result abstain objective abstain abstain objective abstain objective result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain abstain abstain abstain abstain abstain objective method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain other abstain abstain method abstain abstain method abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain result other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain objective objective objective result objective other other other other other