This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt based D ata A ugmentation model ( PromDA ) which only trains small-scale Soft Prompt (i.e., a set of trainable vectors) in the frozen Pre-trained Language Models (PLMs). This avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data. In addition, PromDA generates synthetic data via two different views and filters out the low-quality data using NLU models. Experiments on four benchmarks show that synthetic data produced by PromDA successfully boost up the performance of NLU models which consistently outperform several competitive baseline models, including a state-of-the-art semi-supervised model using unlabeled in-domain data . The synthetic data from PromDA are also complementary with unlabeled in-domain data . The NLU models can be further improved when they are combined for training. Deep neural networks often require large-scale high-quality labeled training data to achieve state-of-the-art performance (Bowman et al., 2015). However, constructing labeled data could be challenging in many scenarios (Feng et al., 2021). In this paper, we study the low-resource Natural Language Understanding (NLU) tasks, including sentence classification and sequence labelling tasks, where only small labeled data is available. Previous works often produce extra labeled data for the NLU models to learn. Wang et al. (2021a) deploys the self-training framework to produce pseudo labelled training data from unlabeled in-domain data which could be expensive to obtain. Xu et al. (2021)  Work done during the internship at Microsoft STCA. has shown that extracting domain-specific unlabeled data from the general corpus is not trivial. Wei and Zou (2019); Dai and Adel (2020) expand the original small training data using automatic heuristic rules, such as randomly synonyms replacement, which effectively creates new training instances. However, these processes may distort the text, making the generated syntactic data grammatically and semantically incorrect. To solve the above dilemma, many existing works (Ding et al., 2020; Yang et al., 2020; Anaby-Tavor et al., 2020) resort to applying Language Models (LMs) or Pre-trained Language Models (PLMs) for data augmentation in a low-resource setting. Given the labeled data, one can directly fine-tune PLMs to generate new synthetic data without additional human effort. However, we argue that, in the low-resource NLU tasks, directly fine-tuning all parameters of PLMs with small training data (especially when there are less than 100 samples) could result in over-fitting and PLMs simply memorizes the training instances. As a result, the generated synthetic data could be very similar to the original training instances and cannot provide new training signals to the NLU models. Recently, several works (Lester et al., 2021; Li and Liang, 2021) propose prompt tuning, which only back-propagates the error to Soft Prompts (i.e., a sequence of continuous vectors prepended to the input of PLMs) instead of the entire model. They show that prompt tuning is sufficient to be competitive with full model tuning while significantly reducing the amount of parameters to be tuned. Thus, the prompt tuning is quite suitable to tackle the above over-fitting issue in low-resource generative fine-tuning, which spawns more novel samples relative to the small labeled data under the premise of ensuring generation quality. allow tuning the additional soft prompts during fine-tuning on the small labeled training data. In addition, we have observed that the initialization of soft prompts has a significant impact on fine-tuning, especially when the low-resource situation reaches an extreme extent. To better initialize the prompt parameters for the data augmentation tasks, we propose task-agnostic Synonym Keyword to Sentence pre-training task to directly pre-train the prompt parameters of PLMs on their pre-training corpora. This task simulates the process of generating entire training sample from partial fragment information (e.g., keywords). Similar to previous works (Ding et al., 2020; Yang et al., 2020; Anaby-Tavor et al., 2020), we could fine-tune PLMs to produce complete synthetic data conditioned on the output tags. We refer this as Output View Generation . To boost the diversity of the generated samples, we introduce another fine-tuning generative task named Input View Generation , which takes the extracted keywords from the sample as the input and the sample as the output. As NLG models trained from small training data still has a certain chance to generate low-quality samples, we leverage the NLU Consistency Filtering (Anaby-Tavor et al., 2020) to filter the generated samples. We conduct experiments on four benchmarks: sequence labelling task CoNLL03 (Tjong Kim Sang and De Meulder, 2003) and Wikiann (Pan et al., 2017), sentence classification task SST-2 (Socher et al., 2013) and RT (Pang and Lee, 2005). Experiment results show that NLU models trained on synthetic data from PromDA consistently outperform several competitive baseline models, including a state-of-the-art semi-supervised NLU models MetaST (Wang et al., 2021a) on Sequence Labelling task. In addition, we find that the synthetic data from PromDA are also complementary with the unlabeled in-domain data . The performance of NLU models can be further improved when both of them are combined. Finally, we conduct diversity analysis and case study to further con-firm the synthetic data quality from PromDA . Our source code is released at https://github. com/GaryYufei/PromDA . Prompt Learning The concept of prompt-based learning starts from the GPT3 model (Brown et al., 2020). Previous works design different prompts to query language models to extract knowledge triples (Petroni et al., 2019) or classify sentences into pre-defined categories (Schick and Schtze, 2021) in the few-shot setting. They construct various discrete prompts manually for these tasks. To reduce the human effort in this selection process, (Gao et al., 2021) proposes to expand prompts using pre-trained language models. However, the selection of discrete prompts is still an independent process and difficult to be optimized together with the downstream tasks in an end-to-end manner. Ben-David et al. (2021) proposes a complicated two-stage model to connect between prompt generation and downstream tasks. To solve this issue, (Lester et al., 2021; Li and Liang, 2021) propose to use soft prompts, which are sets of trainable vectors, in the frozen pre-trained language models. Unlike the hard prompts, these vectors do not correspond to any real words. It allows the optimization with the downstream tasks in an end-to-end manner. As shown in Li and Liang (2021), PLMs with Soft Prompts can often perform better in the low-resource setting. Generative Data Augmentation Hou et al. (2018) generates diverse utterances to improve dialogue understanding models. Xia et al. (2019) uses a bilingual dictionary and an unsupervised machine translation model to expand low-resource machine translation training data. Wu et al. (2019); Kumar et al. (2020) make use of the masking mechanism in many PLM pre-training objective functions (e.g., BERT (Devlin et al., 2019), BART (Lewis et al., 2020)) and produce new synthetic data by masking randomly chosen words in the original training instances. Ding et al. (2020); Yang et al. (2020); Anaby-Tavor et al. (2020) apply LMs and PLMs to learn directly to generate new synthetic data for NLU tasks (i.e., sequence labeling and commonsense inference tasks after trained (fine-tuned) on the relatively large training data. These works often directly apply off-the-shelf LMs or PLMs to generate synthetic data. Wang et al. (2021b) proposes to use unlabelled data as hard prompt to generate synthetic data without any training, limiting its application in complicated NLP tasks. To best of our knowledge, PromDA is the first PLMs with Soft Prompt that are especially designed for the data augmentation task. This section first formulates the data augmentation for low-resource NLU task. We then intro-4243 Figure 1: The Overall of PromDA . duce the three important components in Our proposed Prompt-based Data Augmentation method ( PromDA ), including i) prompt-based learning in pre-trained language models; ii) dual synthetic data generation view and iii) Consistency Filtering . Figure 1 shows the overall of PromDA . In the low-resource NLU tasks, only a set of labeled training data T = { ( x 1 , y 1 ) ,    , ( x n , y n ) } is available where n is relatively small (i.e., less than a hundred). Data Augmentation generates synthetic labeled training data TLM = { ( x 1 ,  y 1 ) ,    , ( x n ,  y n ) } from the original labeled training data T using language models. The goal is that the NLU models trained using T  TLM outperform the NLU models only trained using T . Fine-tuning is the prevalent way to adapt PLMs to specific down-stream tasks (Devlin et al., 2019). However, for low-resource data augmentation, we expect the generated synthetic training data TLM to be different from T and to provide new information for NLU models to learn. A fine-tuned PLM, which is biased towards a small number of training instances, may not be an optimal solution. Prompt-based learning, starting from the zero-shot instructions in GPT3 (Brown et al., 2020), keeps the whole PLMs parameters frozen and only prepends the discrete natural language task instructions (e.g. translate to English) before the task inputs. Freezing the PLMs parameters might help generalization during training. However, finding suitable discrete task introductions cannot be easily optimized in an end-to-end fashion and requires extra human effort. In this paper, inspired by the re-cent work (Lester et al., 2021; Li and Liang, 2021), we replace the task introductions with Soft Prompt (i.e., a sequence of continuous and trainable vec-tors). During training, we only update the parameters of this Soft Prompt and fix all PLMs parameters. We mainly focus on generating synthetic training data using seq2seq Transformer-based PLMs. Unlike Lester et al. (2021) which only prepends Soft Prompt at the input layer, inspired by Adaptor (Houlsby et al., 2019) which adds trainable Multi-layer Perceptron (MLP) at each transformer layer, we prepend a sequence of trainable vectors at each transformer layer. We denote P j = { p j 1 ,    , p jk } as the Soft Prompt at the j th layer. The i th hidden states at the j th layer h ji in the Transformer model is defined as follows: h ji =   p ji i  k w i i > k  j = 0 Trans ( h j  1 ) i Otherwise (1) where Trans () is the forward function the Transformer layer and w i is the fixed word embedding vector at the input layer. Compared to (Lester et al., 2021), this allows gradients to be updated at each layer and better complete the learning tasks. The parameter initialization of the Soft Prompt P has a significant impact on the generated synthetic data quality, especially in the low-resource Data Augmentation task. Lester et al. (2021) proposes to further pre-train the full PLMs parameters, without the prompt parameters, to enhance 4244 Algorithm 1 Dual-View Data Augmentation: Given few-shot labeled dataset T , the number of iteration N ; return a trained NLU model MNLU . the prompt capability. However, this strategy (i.e., full PLM pre-training) introduces significant computation overhead and does not provide any insight about prompt initialization. Instead, we propose to directly pre-train the parameters of the Soft Prompt with the frozen PLMs. Given that data augmentation produces full syntactic data from partial information (e.g., output tags and keywords), we propose Synonym Keywords to Sentence pretraining task. Given a chunk of text, we extract keywords using unsupervised keyword extraction algorithm Rake (Rose et al., 2010). We randomly replace some of these extracted keywords with their synonyms, via WordNet (Fellbaum, 2010). Given these synonym keywords, the Soft Prompt is pre-trained to reconstruct the original text chunks. When applying this Soft Prompt for data augmentation, we only need to fine-tune the Soft Prompt with the few-shot labeled data T . This pre-training process only happens once. We only use the task-agnostic general-purpose pre-training corpus. Previous works often restrict the encoder inputs to fixed keywords or limited labels, such as unconditional generation (Yang et al., 2020) and label-conditional generation (Anaby-Tavor et al., 2020). The relatively small input space could result in similar outputs. To enrich the input space, we propose Dual-View Data Augmentation that generates synthetic data from Input View , which is conditioned on the keywords in the input sentences, and Output View , which is conditioned on the output labels. Table 1 shows examples of these two views. As illustrated in Algorithm 1 (line 2 to 7), after fine-tuning the Soft Prompt in PLMs, PromDA first generates T 1 I and T 1 O from Input View and Output View , respectively. PromDA then extracts output labels from T 1 I and keywords from T 1 O . These new output labels and keywords are fed into the Output View and Input View in MLM to generate another two sets of new synthetic data T 2 O and T 2 I . In this way, the resulting output text should maintain a higher level of diversity and include more novel words/phrases/knowledge. Dual View via Prompt Ensemble Ensembles of different neural models can often achieve better performance (Hansen and Salamon, 1990). Prompt-based learning provides an efficient way to model ensemble. By training K sets of Soft Prompt , we create K models sharing the same frozen PLMs. In our case, after prompt pre-training, we treat Input View and Output View as two independent models and use the Soft Prompt parameters P to initialize the parameters of P input and P output . During the PromDA fine-tuning, the gradients from the Input View and Output View training instances are only applied to parameters P input and P output , respectively. This prompt ensemble allows the two views to generate synthetic data independently. As a result, the final output should include diverse real-world knowledge. As PromDA is trained from small training data, it is possible to generate low-quality samples. We leverage the NLU Consistency Filtering (Anaby-Tavor et al., 2020) to filter the generated samples. Specifically, given synthetic data with generated labels produced by PromDA , we use the NLU models to label these data again and only keep the instances with consistent outputs from PromDA and the NLU models. As shown in Algorithm 1 (line 8 to 12), M rNLU filters the raw synthetic data  TLM into TLM which are combined with few-shot labeled data T to train new NLU models M r +1 NLU . As M r +1 NLU is generally better than M rNLU , we iterate this process N times to obtain stronger NLU models. This section first introduces experimental setup in Sec 4.1, and then presents main experiment results in Sec 4.2. Sec 4.3 conducts ablation study. In 4245 Sequence Labelling GT: [ Org All Fishermen 's Association] secretary [ Per N.J. Bose] said the strike would continue indefinitely. Sec 4.4, We compare PromDA and unlabeled data, present diversity analysis and a case study. We conduct experiments on Sentence Classification tasks SST2 (Socher et al., 2013) and RT (Pang and Lee, 2005) and Sequence Labeling tasks CoNLL03 (Tjong Kim Sang and De Meulder, 2003) and Wikiann (Pan et al., 2017). For each benchmark, we conduct shot-10, 20, 50, 100 experiment. In ShotK , we sample K labeled instances for each output tag from the full training data. We repeatedly experiments 5 times and report the averaged micro-F1. The Baseline model is BERT-BASE model only trained with few-shot training data T . Given the newly generated synthetic data TLM , we train the same BERT-BASE model using the same set of hyper-parameters. In sequence labeling tasks, we use rule-based data augmentation method SDANER (Dai and Adel, 2020) and MetaST (Wang et al., 2021a), a state-of-the-art self-training method, requiring additional unlabeled in-domain data . For sentence classification tasks, rule-based EDA (Wei and Zou, 2019), Back-Translation ( BackT. ) and bert-based CBERT methods are used. We adapt LAMBADA (Anaby-Tavor et al., 2020) as a PLM-based method for all tasks. Implementation Details PromDA is built on the top of the T5-Large model (Raffel et al., 2020). PromDA requires Prompt Pre-training and fine-tuning with down-stream tasks. In both stages, we use Adafactor optimizer (Shazeer and Stern, 2018) with learning rate 1e-3 and weight decay 1e-5 to train the Soft Prompt parameters. For pre-training, we use the realnewslike split in the T5 pre-training corpus C4 as the input. The pre-training batch size is 72 and we pre-train PromDA for 100k steps. We split the realnewslike dataset into train and development split (i.e., 10000 pages). We will check the PPL on the development split every 5,000 steps. We save the model with lowest PPL. When fine-tuning on the few-shot data T , we set the batch size 32 and we train PromDA for 1,000 steps. We only upgrade the fine-tuning step to 5,000 on the shot-50 and shot-100 for Wikiann and CoNLL03. More experiment setup see Section A in the Appendix. 4.2 Main Results Sequence Labeling Tasks Table 2 summarizes the experiment results in shot-10 and shot-50. In both settings, the performance of NLU models trained with the synthetic data from PromDA are boosted up by a large margin (i.e., 4.8% and 7.5% for CoNLL03 and Wikiann, respectively). PromDA also outperforms rule-based SDANER and fully fine-tuned PLM LAMBADA methods. In general, PLM-based approaches produce better synthetic data than SDANER does. Surprisingly, the NLU models supported by PromDA achieve slightly better performance than MetaST which uses unlabeled in-domain data . This shows that PromDA could potentially reduce extra human effort in collecting unlabeled in-domain data for the low-resource NLU tasks. Figure 2 shows the performance in the shot-{10, 20, 50, 100} settings. The NLU models supported by PromDA consistently outperform other systems in all settings. Compared to Wikiann, the improvement margin in CoNLL03 is smaller. This could because the performance of CoNLL03 baseline is relatively high. Sentence Classification Tasks Table 3 shows the experiment results in shot-10 and shot-50. Similar to the results in the sequence labeling tasks, adding the synthetic data from PromDA significantly boosts up the performance of NLU models (more than 10% in both benchmarks in shot-10). PromDA also outperforms various competitive methods, including BackT. , CBERT and LAMBADA . Although LAMBADA has higher level of flexibility and generates synthetic data from output tags, it only performs similar to CBERT . This could be because of the over-fitting issues when fine-tuning with small training data. Prompt-empowered PromDA successfully avoids this issue and produce high-quality synthetic data to support the NLU model training. Figure 2 shows the performance in the shot-{10, 20, 50, 100} settings. NLU models supported by PromDA consistently outperform all other systems in all setups. Discussion LAMBADA performs consistently worse than PromDA (e.g., more than 10% F1 score gap in the SST2 and RT experiment). This is because fully fine-tuned PLMs can easily memorize the limited labeled training data and produce similar synthetic data. In contrast, the prompt-based learning allows PromDA to maintain high generalization ability and provide new training signals to the NLU models. The results from PromDA are all statistical significant, compared to the Baseline model (paired student's t-test, p < 0.05). We conduct ablation study for the components Prompt Pre-training , Dual-View Data Augmentation and Consistency Filtering on the CoNLL03 and SST2 Benchmark under the shot-10 setting. Prompt Pre-Training In No PT , we directly fine-tune two separated PLMs to learn the Input View and Output View . In No PT Pre-Training , we remove the Prompt Pre-training Task ( Synonym Keywords to Sentence ). In Full Pre-Training , we apply the Prompt Pre-training Task to fine-tune the whole PLMs parameters. Finally, in LM Adaptation : we replace PromDA with solution in Lester et al. (2021). As shown in Table 4, the fully fine-tuned PLMs ( No PT ) performs worse than our proposed PromDA method (4.6% F1 score lower), showing the positive contribution of Soft Prompt for low-resource NLU Data Augmentation. Further, removing PT Pre-training ( No PT Pre-Training ) or applying PT Pre-training to fine-tune all PLMs parameters ( Full Pre-Training ) also delegate the PT Pre-training performance by 3.1% and 6.0% F1 score, respectively, showing the importance of using PT Pre-training to learn a reasonable prompt initialization. Similarly, LM Adaptation also fine-tunes the whole PLMs and achieves similar performance as Full Pre-Training . It is recommended to directly train the prompt parameters. Dual-View Data Augmentation Next, we show the effect of Dual-View Data Augmentation in PromDA . Input Only and Output Only only generate synthetic data via the Input View and Output view , respectively. These two Single-View models generate the same number of synthetic data as the PromDA does. As shown in Table 4, the synthetic data from these two Single-View models 4247 DataSet C03 SST2 Ave. successfully boost up the NLU model performance. However, their corresponding NLU models perform worse than the ones supported by PromDA . This shows that synthetic data from different views provide meaningful and different training signals to the NLU models. Interestingly, NLU models trained on the Output view perform better than the ones trained on the Input View , indicating that output tags are more expressive signals to guide PLMs to generate high-quality synthetic data. Finally, instead of training two views on the separated prompt parameters, we train two views on the same prompt parameters in Single Prompt . The NLU models trained on Single Prompt synthetic data perform worse than the NLU models supported by PromDA , showing the importance of Prompt Ensemble for Dual-View Data Augmentation . Setup w/o Filtering Iter-1 Iter-2 Iter-3 C03 72.0 76.7 77.6 77.5 SST2 69.2 77.5 79.7 81.4 Table 5: Ablation Study For Iteration-based NLU Consistency Filtering. Consistency Filtering Finally, we examine the effect of Consistency Filtering in PromDA . In table 5, we show the NLU model performance without any filtering ( w/o Filtering ) and with k iteration ( Iter-1 , Iter-2 and Iter-3 ). The filtering has an important effect on the NLU performance. Without removing low-quality synthetic data, the performance gap almost disappears. The iteration filtering also has a positive effect on the NLU performance. In particular, in the SST2 Benchmark, the NLU model performance increases ~4% F1 score after three iterations. PromDA with T5-Base We verify whether PromDA could work with different pre-trained language models. We replace the T5-Large model with the T5-base model. The new PromDA can also improve the few-shot baseline models by a large margin. On the SST2 shot-10 setup, the NLU model is improved from 66.1 to 76.3 F1 score, which also beats other models presented in Table 3. PromDA in the high-resource setting To show the advantages of PromDA in the high-resource setting, We replace the few-shot training data with the full training data. We find that PromDA can still improve the baseline model performance. In SST2, after adding syntactic data, the NLU performance is improved from 90.8 to 92.3 F1 score. Improvement Margin Difference As shown in Table 2 and 3, the improvement margins in the sentence classification tasks (i.e., more than 15% F1 score) are generally larger than the ones in the sequence labelling tasks (i.e., less than 10% F1 score). This could because i) the sequence labelling task is a more fine-grained and knowledge-intensive task than the sentence classification task; ii) the synthetic data for the sequence labelling tasks includes entity type and boundary, which is more challenging for PLMs to generate, in particular for low-resource settings, compared to the sentence classification task. PromDA and Unlabeled Data The above experiments are based on the assumption that no unlabeled data is available. In this section, we explore the connection between PromDA and unlabeled data . To incorporate unlabeled data into our NLU 4248 Sequence Labeling GT: It quoted an [ Org Interior Ministry] statement as saying [ Per Shabir Ahmad Muhammad Jalil] was executed in [ Loc Mecca] . models, we apply the classic self-training framework (Scudder, 1965) to the NLU models. Specifi-cally, for each unlabeled instance, we use the NLU models to label it and record the output tags and corresponding likelihood score. The low likelihood score means predictions with less confidence. We rank all unlabeled instances based on the likelihood score and remove instances at the bottom 20%. Table 6 shows the experiment result of four benchmarks under the shot-10 setting. The Effect of Unlabeled Data Domain We design three settings: Unlabeled In-domain Data ( UID ), Unlabeled Near-domain Data ( UND ) and Unlabeled General-domain Data ( UGD ) where the unlabeled data come from exactly same , similar and general-purpose domains. We exchange the training data between CoNLL03 and Wikiann, and between SST2 and RT to simulate similar domains. We randomly sample sentences from PLM pre-training corpus to simulate the general-purpose domain. We note that unlabeled data domain has a great impact of the self-training performance. Even a slight domain shift (i.e., UND ) delegates the NLU performance by 2.5%. The performance of NLU models trained with unlabeled data from general-purpose corpus are even 3.2% lower than the NLU baseline models only trained with few-shot labeled data T . Both sequence labeling tasks 4249 and sentence classification tasks follow this trend, but sequence labeling tasks is more sensitive to the unlabeled data domain. Extra human effort is still required, for semi-supervised learning, to select suitable domains to collect unlabeled data. Combining Unlabeled In-domain Data with PromDA We apply the above self-training algorithm to the final NLU models ( PromDA ) supported by PromDA with unlabeled in-domain data . The resulting NLU models are further improved, on average, by 2.0% ( w/ UID in the last row). More sophisticated semi-supervised learning algorithms may introduce more improvement. This shows that a) synthetic data from PromDA and unlabeled in-domain data provide different information to the NLU models; b) PromDA successfully extracts the embedded knowledge in the PLMs and presents them in the generated synthetic data. Diversity Analysis In Table 8, we show the diversity of the generated synthetic data from PromDA and other baseline models. We sample 10 new synthetic data from each training instance. We use Novel Mention (number of entity mentions or keywords not appearing in the training data) and Self-BLEU score (Zhu et al., 2018) to measure the diversity. In general, simple generative data augmentation approaches (i.e, BackT. and CBERT ) can easily produce Novel Mentions, but their generated synthetic data lacks diversity (rel-atively low self-BLEU score). The prompt-based learning helps PromDA to produce the most diverse synthetic data with the most Novel Mentions in both benchmarks. Due to the over-fitting issues, LAMBADA produces synthetic data that are less or equal diverse than other baseline approaches. Interestingly, the NLU models trained on these synthetic data achieve the second best performance. This could because LAMBADA coherently generate the whole synthetic sentences, while others reply on the random and/or heuristic rules. Synthetic Data Case Study Table 7 shows representative examples generated by our proposed PromDA and methods. In the Sequence Labelling example, the rule-based SDANER shuffles the original word order and creates low-quality text. The LAMBADA model generates a new synthetic instance by modifying three text spans in the original training instance (e.g., changing statement to newspaper). In contrast, Our PromDA method generates a completely new and reasonable event Model NM  Self-B  F1  CoNLL03 SDANER 141.4 0.770 72.9 LAMBADA 107.6 0.761 75.0 PromDA 351 0.259 77.5 SST2 EDA 59.6 0.889 66.7 BackT. in a bank, as well as correct and novel geographical locations in the generated synthetic data. Similarly, in the sentence classification tasks, LAMBADA naively combines text chunks from two training instances in the second example. PromDA mentions some keywords in the training data, but adds more information into the output. In another example, PromDA comments on a screenwriter (not appearing in the training data) with a sequence of coherent words. Finally, PromDA successfully moves the topic from the film The Saigon of 1952 to the Saigon in 70s. In summary, PromDA can extract the embedded real-world knowledge from the PLMs and introduces these knowledge into a relatively long sentence in a fluent way. In this paper, we present the first prompt-based pre-trained language model PromDA for low-resource NLU data augmentation. Experiments on four benchmarks show the effectiveness of our proposed PromDA method. In the future, we plan to expand PromDA to other NLP tasks, including question answering, machine reading comprehension and text generation tasks. We thank anonymous reviewers for their insightful suggestions to improve this paper. Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng and Daxin Jiang are supported by Microsoft Software Technology Center at Asia (STCA). Yufei Wang also receives a MQ Research Excellence Scholarship and a CSIRO's DATA61 Top-up Scholarship.
method objective abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result objective abstain method abstain method method method abstain result abstain method other abstain other other other other other other other other other other other other other other other other objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method method method method abstain method method abstain method abstain abstain method method method method method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective abstain other other other