Most previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines (i.e., pretrained models with over one billion parameters). Under this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to generate label-flipped data. Central to the idea of FlipDA is the discovery that generating label-flipped data is more crucial to the performance than generating label-preserved data. Experiments show that FlipDA achieves a good tradeoff between effectiveness and robustnessit substantially improves many tasks while not negatively affecting the others. 1 1 Introduction Data augmentation is a method to augment the training set by generating new data from the given data. For text data, basic operations including replacement, insertion, deletion, and shuffle have been adopted widely and integrated into a wide range of augmentation frameworks (Zhang et al., 2015; Wang and Yang, 2015; Xie et al., 2020a; Kobayashi, 2018; Wei and Zou, 2019). Generative modeling methods such as back-translation have also been employed to generate augmented samples (Fadaee et al., 2017; Sennrich et al., 2016). However, there are two major limitations. First, some general augmentation methods are based on weak baselines without using large-scale pretrained language models. Recent work showed that some of  The authors have contributed equally to this work. the data augmentation methods are less useful when combined with large pretrained models (Longpre et al., 2020). Second, most prior studies are carried on simple tasks such as single-sentence classification where it is easier to generate legit augmented samples. For harder tasks such as natural language inference (e.g., telling whether sentence A entails sentence B), it is not clear whether previous methods still help. This work takes a step further to study data augmentation under strong baselines and hard tasks. Our study employs large-scale pretrained language models such as DeBERTa (He et al., 2020c) with over one billion parameters as baselines. Moreover, we target a very challenging settingfew-shot natural language understanding (NLU). Following (Schick and Schutze, 2021), we consider challenging NLU tasks including question answering, textual entailment, coreference resolution, and word sense disambiguation, and use only 32 training examples for each task. Under this setting, we reproduced several widely-used prior methods for data augmentation. Our experiments lead to two unexpected discoveries: (1) most of prior augmentation methods bring only marginal gains at best and are not effective for most tasks; (2) in many cases, using data augmentation results in instability in performance and even entering a failure mode; i.e., performance may drop by a lot or fluctuate severely depending on which pretrained model is used. The above issues prevent these augmentation methods from practical usage for few-shot learning. We propose a novel method FlipDA that achieves both effectiveness and robustness for hard few-shot tasks. Preliminary experiments showed that label-flipped data often largely improve the generalization of pretrained models, compared to augmented data that preserve the original labels. Based on this observation, FlipDA first generates data using word substitution based on a pretrained T5 (Raffel et al., 2020) and uses a classifier to select label-flipped 8646 data. Experiments demonstrate FlipDA substantially improves performance on many of the hard tasks, outperforming previous augmentation baselines in terms of average performance by a large margin. Moreover, FlipDA is robust across different pretrained models and different tasks, avoiding failure modes. Data Augmentation. An important type of augmentation methods are based on word substitution , such as synonym replacement (Zhang et al., 2015), KNN replacement (Wang and Yang, 2015; Vija-yaraghavan et al., 2016), Unif replacement (Xie et al., 2020a), TF-IDF replacement (Xie et al., 2020a), Bi-RNN replacement (Kobayashi, 2018), and other entity replacement methods (Raiman and Miller, 2017; Miao et al., 2020; Yue and Zhou, 2020) etc. EDA (Wei and Zou, 2019) combines four simple augmentation methods and back translation (BT) (Fadaee et al., 2017; Sennrich et al., 2016; Yu et al., 2018) is also widely used. Unfortunately, EDA and BT are shown to be less useful with large pretrained models (Longpre et al., 2020). Some augmentation methods are based on the perturbation in the feature space (Zhang et al., 2018a; Guo et al., 2020; Chen et al., 2020b,a; Miao et al., 2020; Kumar et al., 2019). Generation (Xia et al., 2020; Li et al., 2019; Yoo et al., 2019; Ng et al., 2020; Liu et al., 2020; Hou et al., 2018) based methods are also proposed for better data diversity. In addition, large pretrained models have been used for data augmentation. (Kumar et al., 2020) utilize large pretrained models, such as GPT-2, BERT, and BART, for conditional data augmentation. LAMBADA (Anaby-Tavor et al., 2020) finetunes a GPT-2 model with the priming technique to get augmented examples. GPT3Mix (Yoo et al., 2021) uses GPT-3 along with prompting to generate augmented data for classification tasks. Our method is similar to this line of work in that we also use pretrained models for generating augmented data. However, there are the following key differences. First, it is challenging for these prior methods to handle long sequences or multiple sentences. In our preliminary experiments, we were not able to use these methods to generate proper data samples (see details in Section 4). Second, besides generating augmented samples, we found it crucial to use label-flipped data for augmentation, which is a unique and critical aspect of FlipDA. Self-training. Self-training (III, 1965) iteratively augments training data by labeling unlabeled data with a trained model (Yarowsky, 1995; Riloff, 1996). Knowledge distillation and pseudo-labeling are special forms of self-training (Hinton et al., 2015; Lee et al., 2013; Reed et al., 2015). Strong data augmentation (Zoph et al., 2020), equal-or-larger model (Xie et al., 2020b), additional noise (Xie et al., 2020b; He et al., 2020a), and feedback of the student's performance (Pham et al., 2020) are helpful for self-training. Self-training bears similarity to the second phase of FlipDA where a teacher model is used to filter samples. Different from self-training, FlipDA leverages the advantages of label flipping to improve performance and does not rely on unlabeled data. Label Flipping. Our manual label flipping augmentation procedure is analogous to (Kaushik et al., 2020) and (Gardner et al., 2020). Kaushik et al. (2020) aimed to mitigate the effects of learning spurious features. Gardner et al. (2020) targeted reducing systematic gaps in the dataset. In contrast, we target improving few-shot generalization. Moreover, we measure the performance on an existing i.i.d. test set while Kaushik et al. (2020) and Gardner et al. (2020) created more challenging test sets. Most importantly, we propose an automatic method of label flipping, going beyond manual efforts. Contrastive Learning. FlipDA is connected to contrastive learning (CL) (He et al., 2020b; Chen et al., 2020c) in that they both improve generalization by considering label differences. CL uses data augmentation to generate positive instances and uses samples existing in the dataset as negative samples, while FlipDA shows that negative samples can be automatically generated. While previous work on CL focuses on training with large datasets, our experiments show that augmenting a small dataset can improve few-shot generalization. It could be intriguing to see whether such a connection might lead to advances in both fields, e.g., generating negative samples for large-scale contrastive pretraining. Few-Shot NLU Tasks. This work considers a collection of difficult NLU tasks from SuperGLUE (Wang et al., 2019) that require in-depth understanding of the input in order to obtain 8647 high performance, including coreference resolution (Levesque et al., 2011), causal reasoning (Gordon et al., 2012), textual entailment (de Marneffe et al., 2019; Dagan et al., 2005), word sense disambiguation (Pilehvar and Camacho-Collados, 2019), and question answering (Clark et al., 2019; Khashabi et al., 2018; Zhang et al., 2018b). Following Schick and Schutze (2021), we used only 32 training examples to construct a few-shot setting to further increase the difficulty. Large-Scale Pretrained Models. Our setting assumes a large-scale pretrained language model (De-vlin et al., 2019; Lan et al., 2020; He et al., 2020c) is available and few-shot learning is performed based on the pretrained model. This setting is crucial since previous studies found that using a strong pretrained model as the baseline eliminates the ben-efits of data augmentation (Longpre et al., 2020) while large pretrained models are becoming more and more available. Our main result is based on DeBERTa (He et al., 2020c) with over one billion parameters. We also provide results with ALBERT which has fewer parameters (Lan et al., 2020). Our preliminary experiments with a large number of previous methods (in Section 4) lead to a conclusion that there is not an effective and robust method available for this hard setting. We will discuss how we tackle this challenge by proposing a novel data augmentation method FlipDA in later sections. We propose key desiderata for data augmentation methods under the setting of few-shot learning. 1. Effectiveness. A data augmentation method should be able to improve performance on certain tasks in a significant manner. 2. Robustness. A data augmentation method should not suffer from a failure mode in all cases. Failure modes are common for few-shot learning where some minor changes might cause substantial performance drop. We argue this should be used as a key evaluation metric. We consider two types of robustness: (1) robustness w.r.t. different base pretrained models and (2) robustness w.r.t. various tasks. Since previous methods are not sufficiently effective and robust in our preliminary experiments (see Tables 5 and 6 in Section 4 for details), we use manual augmentation to investigate what kind of augmented data is beneficial for large pretrained models in the few-shot setting. We mainly study two types of data augmentationone that preserves the labels and the other that flips the labels. Since manual augmentation is time consuming, we select a subset of representative SuperGLUE tasks here. To augment label-flipped data, the following principle is appliedmaking minimal changes to the original text sample to alter the label. Augmentation includes word addition, deletion, and substitution. To augment label-preserved data, we substitute some of the words with semantically similar words but make sure that the label is unchanged. Results are shown in Table 1. 2 Flipping labels substantially improves performance on three of the tasks by up to 10 points, while preserving the labels only has minor gains. In contrast, many prior methods on data augmentation focus on creating data examples that are assumed to have the same labels as the original ones. This might explain why previous augmentation methods are not sufficiently effective for the few-shot setting. Some of the label-flipped augmented examples are shown in Table 2. We conjecture that label flipping augmentation provides useful information about the important components in a sentence that determine the label. In other words, augmented samples provide intermediate supervision that explains the predictions, improving generalization in a few-shot setting. There is a caveat about this manual augmentation experiment. Although we follow certain principles and pay much attention to the augmentation quality, the manual augmentation procedure is inevitably 2 For each original example, we produce one augmented example for each type. The augmented data and the original data are combined for training. Following Schick and Schutze (2021), we train each pattern with three seeds and ensemble these (pattern, seed) pairs. We repeat this ensemble process 3 times and report their mean and standard deviation. subjective and hard to reproduce. For reference, we will make our manually augmented dataset publicly available. More importantly, we will design an automatic method (FlipDA) in the following sections for objective evaluation and reproducibility. We also analyze why augmentation methods usually suffer from failure modes. Most augmentation methods are based on a label preserving assumption, while it is challenging for automatic methods to always generate label-preserved samples. We first examine the samples generated by prior automatic methods EDA (Wei and Zou, 2019) and KNN (Wang and Yang, 2015) in Table 4. In the first example, the keyword rabies is deleted, which not only results in a grammatically incorrect expression but also eliminates the key information to support the hypothesis. In the second example, the Lake Titicaca is replaced by Lake Havasu, which results in a label change from entailment to non-entailment. If a model is trained on these noisy augmented data with the label preserving assumption, performance degradation is expected. We further experimented with EDA (Wei and Zou, 2019) on the RTE task (Dagan et al., 2005) to verify the cause of failure modes. Using EDA decreases the performance by a few percentage points with both ALBERT and DeBERTa, entering a failure mode. We identified two types of noise in the augmented samples: (1) grammatical errors that lead to the difficulty of understanding and (2) mod-ification of key information that alters the labels. We experimented with (1) replacing these noisy samples with the original ones and (2) correcting the labels of the noisy samples. 3 As Table 3 shows, 3 For label correction, if a sample has severe grammatical mistakes and is not understandable by human, we always mark it as not entailment. This is related to an interesting phenomenon that label flipping is usually asymmetric for NLU tasks. We will discuss more of the phenomenon in Section 4.5. both replacing and correcting noisy samples largely improve performance to prevent the failure mode. Moreover, correcting the labels brings large gains, indicating label flipping tends to alleviate the issue. To reiterate, these experiments involve subjective factors and are merely meant to show the intuition of FlipDA, rather than proving its superiority. Observations in Sections 3.3 and 3.4 show that label-flipping could benefit few-shot NLU in both effectiveness and robustness. Reducing grammatical errors is also key to preventing failure modes. This motivates our development of FlipDA that automatically generates and selects label-flipped data without label-preserving assumption. FlipDA consists of 4 steps as shown in Figure 1: 1. Train a classifier (e.g., finetuning a pretrained model) without data augmentation. 2. Generate label-preserved and label-flipped augmented samples. 3. Use the classifier to select generated samples with largest probabilities for each label. 4. Retrain the classifier with the original samples and the additional augmented samples. Formally, given a few-shot training set { ( x i , y i ) } i where x i is text (possibly a set of text pieces or a single piece) and y i  Y is a label. We finetune a pretrained model f to fit the conditional probability for classification f ( x, y ) =  p ( y | x ) . In the second step, we generate augmented samples 8649 Table 4: Augmented example with wrong labels. from the original ones. For each training sample x i , we generate a set of augmented samples S i = {  x i, 1 ,  x i, 2 ,    } . In our implementation, we first use a cloze pattern (Schick and Schutze, 2021) to combine both x and y into a single sequence, and then randomly mask a fixed percentage of the input tokens. This is followed by employing a pretrained T5 model (Raffel et al., 2020) to fill the blanks to form a new sample x (cid:48) (see Appendix A.3 for more details). We find it beneficial to remove the sample if T5 does not predict y given x (cid:48) . Note that using T5 to generate augmented samples does introduce additional knowledge and reduce grammatical errors, but naively using T5 for augmentation without label flipping and selection does not work well (see ablation study in Section 4). After generating the augmented samples, we use the classifier f for scoring. Specifically, let S i be a set of augmented samples generated from the original sample ( x i , y i ) . which contains all augmented samples with y (cid:48) being highest-probability class. Given the set S i,y (cid:48) , we select the sample with the highest predicted probability x (cid:48) , y (cid:48) = arg max x  S i,y (cid:48) ,y = y (cid:48)  p ( y | x ) where x (cid:48) is a sample in the generated set, y (cid:48) is the flipped label, and the estimated probability  p ( y (cid:48) | x (cid:48) ) scored by the model f is the largest in S i,y (cid:48) . After selecting the label-flipped example ( x (cid:48) , y (cid:48) ) , we add ( x (cid:48) , y (cid:48) ) to the augmented training set. In other words, we only add an example into the training set if the model f considers the flipped label to be correct. We apply this procedure to each possible label y (cid:48) (cid:54) = y i . In case S i,y (cid:48) is empty, we do not add any examples to the training set. In practice, we 8650 find it beneficial to also add the example with the highest probability of label preserving, using the same procedure. After augmenting the training set, we retrain the classifier f to obtain the final model. Baselines. We take seven augmentation methods as the baseline, including Synonym Replacement (SR) (Zhang et al., 2015), KNN Replacement (KNN) (Wang and Yang, 2015), Easy Data Augmentation (EDA) (Wei and Zou, 2019), Back Translation (BT) (Fadaee et al., 2017), TinyBERT (T-BERT) (Jiao et al., 2019), T5-MLM, and MixUP (Zhang et al., 2018a). For more details about baseline selection and implementation, please refer to Appendix A.2. Evaluation Protocol We evaluate augmentation methods based on PET (Schick and Schutze, 2021). Following PET, we take a set of pre-fixed hyper-parameters (see Appendix A.1). Considering few-shot learning is sensitive to different patterns and random seeds (Dodge et al., 2020; Schick and Schutze, 2021), we reported the average performance over multiple patterns and 3 iterations. We evaluate FlipDA on 8 tasks with 2 pretrained models. For effectiveness, we use exactly the same metrics (i.e., accuracy, F1, and EM) as PET (Schick and Schutze, 2021). For robustness, we propose a new metric MaxDrop (MD), which measures the maximum performance drop compared to not using augmentation over multiple tasks for a given method. Given tasks t 1 ,..., t n , a target method M , and a baseline method MB , MD is defined as MD= max t { t 1 ,...,t n } max(0 , score t,M B  score t,M ) , where score t,M ( score t,M B ) denotes the performance of method M ( MB ) on task t . Smaller values indicate better robustness w.r.t tasks. Results are presented in Table 5 and Table 6. We observe that FlipDA achieves the best performance among all data augmentation methods in both effectiveness (Avg.) and robustness (MD) on both ALBERT-xxlarge-v2 and DeBERTa-v2-xxlarge. Specifically, FlipDA achieves an average performance of 74.63 on ALBERT-xxlarge-v2 and an average of 80.23 on DeBERTa-v2-xxlarge, both of which outperform baselines by around 3 points. It suggests FlipDA is effective in boosting the performance of few-shot tasks by augmenting high-quality data without causing too many side effects. FlipDA shows improvements on all tasks except WSC, while all the other methods only work on a few tasks (denoted with underlines). Such observations are consistent with the MaxDrop results, where FlipDA achieves the lowest MaxDrop value of 0.0 on ALBERT-xxlarge-v2 and 1.28 on DeBERTa-v2-xxlarge. This implies FlipDA is robust to different types of tasks, while other augmentation methods could only be effective for partial tasks and not sufficiently robust. Effectiveness of Pattern-based Data Cloze To study different methods of obtaining candidate augmented data, we feed candidates obtained by different methods into the same classifier (as FlipDA uses). Table 6 shows the ablation results. FlipDA outperforms all the other baseline methods with a classifier (i.e., with FlipDA cls). Other methods of obtaining augmented data candidates cannot reach similar performance as FlipDA when combining with FlipDA classifier, which proves the effectiveness of our pattern-based data cloze strategy with T5. Reasons could be that T5-based augmentation produces samples with fewer grammatical errors. (will further discuss in Sec 4.7). Moreover, T5-style blank filling could produce samples that are more compatible with label flipping. Effectiveness of FlipDA Classifier We then compare the performance of different methods with and without the FlipDA classifier. According to Table 6, most baseline methods with the FlipDA classifier outperform the original version in terms of both effectiveness (Avg.) and robustness (MD). This demonstrates that the FlipDA classifier which is capable of flipping labels and filtering data is effective in augmenting high-quality data and improving few-shot NLU performance. The only exception is BT-6. The reason could be data augmented by back translation usually lack diversity and are less likely to change labels, and using the FlipDA classifier further decreases diversity and hurts its performance. The improvement brought by the FlipDA classifier is more consistent on BoolQ, RTE, and MultiRC. This may be because these tasks involve predicting a single token with two opposite choices, and thus label flipping might happen more often. Some of the other tasks such as COPA and WSC in-8651 Table 5: Performance of baseline methods and FlipDA based on PET and ALBERT-xxlarge-v2 (baseline denotes the original PET with no data augmentation. Underline denotes values that outperform baseline. Bold denotes the best-performed ones of the task). volve predicting multiple tokens, which makes generating label-flipped data more difficult. This leads to less substantial improvement on these tasks. A follow-up question is how label-flipped data and label-preserved data respectively contribute to the overall improvements. We run decoupling label-flipped data and label-preserved data. Results are in Table 7, where bold text represents the best-performed methods. We conclude that augmenting both label-flipped and label-preserved data leads to the best average performance. Besides, values with underlines denote the second-best performance, most of which are augmenting only label-flipped data. Augmenting only label-preserved data leads to the worst performance, even slightly underperforming the non-augmentation baseline. This demonstrates the high effectiveness of label-flipping. This aligns well with our analysis in Section 3.3. More results on ALBERT are in A.7.2. Section 4.4 proves that label-flipped augmented data are more effective in improving few-shot performance than label-preserved ones. It is even more intriguing to study which direction of label flipping is able to benefit the few-shot performance to the maximum extent. We experiment with 4 binary 8652 Table 7: Ablation study on label-flipped data classification tasks, i.e., RTE, BoolQ, WiC, and MultiRC. Each task has 4 directions of label transformation. We conduct experiments that augment data in each of the four directions respectively and compare their effectiveness. Results on DeBERTa are shown in Table 8, and results on ALBERT are in Appendix A.7.3. We can see that some tasks are asymmetric, i.e., transforming in one direction is more beneficial than the other, such as BoolQ, RTE, and WiC. We conjecture that it is because it is relatively easy for a model to generate samples with answers in some direction (from yes to no in BoolQ, from 'en-tailment' to not entailment in RTE, and so on). While some tasks are symmetric, i.e., the difference between the two directions is not significant, such as MultiRC. On all tasks, even though some direction is better than others, augmenting with only one direction will affect the label distribution. This will likely lead to a lower performance than the baseline. Augmenting with all directions is still necessary for the best performance. We propose four plausible strategies for augmented data selection, and quantitatively evaluate them. The four strategies are described as follows. 2. Global Top K . For each label transformation direction, all the candidate augmented data are gathered and sorted by their predicted probabilities, and the topK ( or topr % ) samples with the highest probabilities are selected. 3. Global Top P . Similar to Global Top K , but augmented data with predicted probabilities higher than a threshold P are selected. 4. Diverse Top K . Similar to Global Top K except that a mechanism is used to balance between the original samples. Concretely, we first select the top-1 augmented samples of each original sample (ranked by decreasing probabilities), and then select the top-2, top-3, etc, until K samples have been selected. Since FlipDA can be viewed as a self-training algorithm, we also add a self-training algorithm Noisy Student (Xie et al., 2020b) as another baseline. We treat the augmented data as unlabeled data and add noises with a dropout rate of 0.1. Table 9 shows the results of different strategies on different tasks. More results on ALBERT are in A.7.4. For Global Top P , we set the threshold P at 0.9 or 0.95, whichever is better. For Global Top K and Diverse Top K , we select the top 10% or 20% augmented examples, whichever is better. Our strategies outperform Noisy Student. Among our four data selection strategies, the Default strategy and Diverse Top K perform the best. Both methods emphasize diversity by using augmented data from different samples. This demonstrates the importance of data diversity and balance for augmented data selection. We show four augmented cases on the RTE task by FlipDA in Table 10. Please refer to Appendix A.8 for more augmented examples. In the first case, we can see that the T5-model changes the name of the tropical storm from Debby to Maria, and it also changes the trop-ical storm to its hypernym hurricane, and all these changes contribute to a different expression 8653 Table 9: Results of different strategies for choosing augmented data on DeBERTa (xxlarge). without affecting its label. The second case adds not to the premise and therefore the label flips. The third case changes dwindles to its antonym increased, and then the label changes from Not Entailment to Entailment. The last case changes the future tense to the simple past tense, April to March, and May to April correspondingly, without affecting its label. We can see that the way to change or keep the label is rich and natural. Moreover, the generation quality is improved compared to cases generated by EDA in Table 4, which also addresses the concerns of generation quality raised in Section 3.4. We propose to study few-shot NLU based on large-scale pretrained models. Two key desiderata, i.e., effectiveness and robustness, are identified. Based on the empirical insight that label flipping improves few-shot generalization, we propose FlipDA with automatic label flipping and data selection. Experiments demonstrate the superiority of FlipDA, outperforming previous methods in terms of both effectiveness and robustness. In the future, it will be crucial to theoretically understand why and how generating label-flipped data in the neighborhood of existing data points improves generalization. Moreover, increasing the diversity and quality of augmented data generation is also an important long-term goal. Zhou and Li are supported in part by the National Natural Science Foundation of China Grant 62161146004, Turing AI Institute of Nanjing and Xi'an Institute for Interdisciplinary Information Core Technology. Tang is funded by NSFC for Distinguished Young Scholar (61825602). Zheng is Funded by China Postdoctoral Science Foundation (2021M690471).
abstain objective result objective abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain method method abstain method method result abstain objective abstain abstain abstain abstain other other other other other other other other other other objective other other abstain objective other other other other other other other abstain other other objective method objective other other other objective other abstain other method abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain method abstain abstain method method method method method abstain abstain abstain method abstain method method method method method method abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method abstain abstain result abstain abstain abstain abstain abstain abstain result abstain objective abstain objective abstain abstain abstain other other other