Natural language processing (NLP) algorithms have become very successful, but they still struggle when applied to out-of-distribution examples. In this paper we propose a controllable generation approach in order to deal with this domain adaptation (DA) challenge. Given an input text example, our DoCoGen algorithm generates a domain-counterfactual textual example (D-CON )  that is similar to the original in all aspects, including the task label, but its domain is changed to a desired one. Importantly, DoCoGen is trained using only unlabeled examples from multiple domains  no NLP task labels or parallel pairs of textual examples and their domain-counterfactuals are required. We show that DoCoGen can generate coherent counterfactuals consisting of multiple sentences. We use the D-CON s generated by DoCoGen to augment a sentiment classifier and a multi-label intent classifier in 20 and 78 DA setups, respectively, where source-domain labeled data is scarce. Our model outperforms strong baselines and improves the accuracy of a state-of-the-art unsupervised DA algorithm. 1 1 Introduction Natural Language Processing (NLP) algorithms are constantly improving and reaching significant milestones (Devlin et al., 2019; Raffel et al., 2020; Brown et al., 2020). However, such algorithms rely on the availability of sufficient labeled data and the assumption that the training and test sets are drawn from the same underlying distribution. Unfortunately, these assumptions do not hold in many cases due to the costly and labor-intensive data labeling process and since text may originate from many different domains. As generalization in low resource regimes and beyond the training distribution are still fundamental NLP challenges,  Both authors equally contributed to this work. 1 Our code and data are available at https://github. NLP algorithms significantly degrade when applied to such scenarios. Domain adaptation (DA) is an established field of research in NLP (Roark and Bacchiani, 2003; Daum III and Marcu, 2006; Reichart and Rap-poport, 2007) that attempts to explicitly address generalization beyond the training distribution (2). DA algorithms are trained on annotated data from source domains to be effectively applied in various target domains. Indeed, DA algorithms have been developed for multiple NLP tasks throughout the last two decades (Blitzer et al., 2006, 2007; Glorot et al., 2011; Rush et al., 2012; Ziser and Reichart, 2017, 2018a,b; Han and Eisenstein, 2019). A natural alternative to costly human annotation would be to automatically generate labeled examples for model training. Doing so may expose the model to additional training examples and better represent the data distribution within and outside the annotated source domains. Unfortunately, generating labeled textual data is challenging (Feng et al., 2021), especially when the available labeled data is scarce. Indeed, labeled data generation has hardly been applied to DA (2). To allow DA through labeled data generation, we present DoCoGen , an algorithm that generates domain-counterfactual textual examples (D-CON s). In order to do that, DoCoGen intervenes on the domain-specific terms of its input example, replacing them with terms that are relevant for its target domain while keeping all other properties fixed, including the task label. Consider the task of sentiment classification (top example in Table 1). When DoCoGen encounters an example from the Kitchen domain (its source domain), it first recognizes the terms related to Kitchen reviews, i.e., knife and solid . Then, it intervenes on these terms, replacing them with text that connects the example to the Electronics domain (its target domain) while keeping the negative sentiment. rithm (Li et al., 2016; Russo et al., 2020) that is trained using a novel unsupervised sentence reconstruction objective. Importantly, it does not require task-annotated data, or parallel pairs of sentences and their D-CON s. A key component of DoCoGen is the domain orientation vector , which guides the model to generate the new text in the desired domain. The parameters of the orientation vectors are learned during the unsupervised training process, allowing the generation model to share information among the various domains it is exposed to. We focus on two low resource scenarios: Unsupervised domain adaptation (UDA) and any domain adaptation (ADA, Ben-David et al. (2021)), with only a handful of labeled examples available from a single source domain. In both UDA and ADA the model is exposed to limited labeled source domain data and to unlabeled data from several domains. However, in UDA the unlabeled domains contain the future target domain to which the model will be applied, while in ADA the model has no access to the target domain during training. To cope with these extreme conditions, we use DoCoGen to enrich the source labeled data with D-CON s from the unlabeled domains. By introducing labeled D-CON s from various domains, we hope to provide the model with a training signal that is less affected by spurious correlations: Correlations between features and the task label which do not hold out-of-domain (OOD) (Veitch et al., 2021). After a brief evaluation of the intrinsic quality of the D-CON s generated by DoCoGen , we evaluate our complete DA pipeline. We focus on two tasks: Binary sentiment classification of reviews and multi-label intent prediction in information-seeking conversations. In both tasks, we follow the UDA and ADA scenarios, for a total of 12 and 8 sentiment setups, respectively, as well as 30 UDA and 48 ADA intent prediction setups. Our results demonstrate the superiority of DoCoGen over strong DA and textual-data augmentation algorithms. Finally, combining DoCoGen with PERL (Ben-David et al., 2020), a SOTA UDA model, yields new SOTA DA accuracy and stability. We first describe research in our DA setups: UDA and ADA. We then continue with the study of counterfactual-based data augmentation, and, fi-nally, we describe research on counterfactual generation methods. Domain Adaptation (DA) The NLP literature contains several DA setups, the most realistic of which is unsupervised domain adaptation (UDA), which assumes the availability of unlabeled data from a source and a target domain, as well as access to labeled data from the source domain (Blitzer et al., 2006). An even more challenging and potentially more realistic setup is the recently proposed any domain adaptation setup (ADA, Ben-David et al. (2021)), which assumes no knowledge of the target domains at training time. There are several approaches to DA, including representation learning (Blitzer et al., 2006; Ziser and Reichart, 2017) and data-centric approaches like instance re-weighting and self-training (Huang et al., 2006; Rotman and Reichart, 2019). Since the rise of deep neural networks (DNNs), most focus in DA research has been directed to deep representation learning approaches (DReL). One line of DReL work employs an input reconstruction objective (Glorot et al., 2011; Chen et al., 2012; Yang and Eisenstein, 2014; Ganin et al., 2016). Another line employs pivot features, which are prominent to the task of interest and common in the source and target domains (Blitzer et al., 2007; Pan et al., 2010; Ziser and Reichart, 2018b; Ben-David et al., 2020; Lekhtman et al., 2021). We deviate from the DReL approach to DA and propose a data-centric methodology. Contrary to the above works, our approach can be applied to both UDA and ADA. Moreover, unlike previous ADA work, which builds upon multi-source DA, our approach can also perform single-source ADA. Counterfactually Augmented Data (CAD) Textual data augmentation (TDA) is a technique for increasing the training dataset without explicitly collecting new examples. This is achieved by adding slightly modified copies of already existing examples (local sampling) or newly created data (global sampling). TDA serves as a solution for insufficient data scenarios and as a technique for improving model robustness (Xie et al., 2020; Ng et al., 2020). There are rule-based and model-based approaches to TDA. Rule-based methods commonly involve insertion, deletion, swap and replacement of specific words (Wei and Zou, 2019), or template-based paraphrasing (Rosenberg et al., 2021). Model-based methods typically utilize a pretrained language model (PLM), e.g., for replacing random words (Kobayashi, 2018; Ng et al., 2020), or generating entirely new examples 7728 Original, Kitchen : A good knife but Quality Control was poor. from a prior data-distribution (Bowman et al., 2016; Russo et al., 2020; Wang et al., 2021). Other model-based methods apply backtranslation (Edunov et al., 2018) or paraphrasing (Kumar et al., 2019) for local sampling. Another approach within local sampling TDA is to change (only) a specific concept that exists in the original example, creating a counterfactual example. Counterfactually-Augmented Data (CAD) is generated by minimally intervening on examples to change their ground-truth label, that is, perturbing only those terms necessary to change the label (Kaushik et al., 2020). CAD is commonly used to improve generalizability (Kaushik et al., 2020; Sen et al., 2021), however empirical results using CAD for OOD generalization have been mixed (Joshi and He, 2021; Khashabi et al., 2020). In this work, we explore a different type of counterfactuals, namely D-CON s, which are the result of intervening only on the example's domain while holding everything else equal, particularly its task label. For sentiment analysis, we may be, for example, interested in revising a negative movie review, making it a negative airline review. In addition, while CAD is mostly generated via a human-in-the-loop process (Kaushik et al., 2020; Khashabi et al., 2020; Sen et al., 2021), our work focuses on automatic counterfactual generation. Counterfactual Generation controllable generation refers to generation of text while controlling for specific attributes (Prabhumoye et al., 2020). The controlled attributes can range from style (e.g., politeness and sentiment) to content (e.g., keywords and entities) and even topic. Keskar et al. (2019) propose to control the generated text by training an LM on datasets annotated with the controlled attributes, and Meister et al. (2020) modify the model's decoding method. Recently, Russo et al. (2020) introduced a global sampling conditional variational autoencoder (VAE), augmenting text while controlling for attributes such as label and verb tense. However, controlling for the task label is challenging in scarce labeled data scenarios (Chen et al., 2021), since generative models require large amounts of labeled data . Counterfactual generation lies at the intersection of controllable generation and causal inference (Feder et al., 2021a). Only few works deal with counterfactual generation, mostly by intervening on the task label. Wu et al. (2021) train a model on textual examples and their manually generated counterfactuals. Other works present methods for controlling for the text domain and semantics (Wang et al., 2020; Feng et al., 2019), yet they all experiment with short texts, while our model can generate longer texts, consisting of multiple sentences. A recent work by Yu et al. (2021) focuses on generation of new target-domain examples for aspect-based sentiment analysis (ABSA) (Pon-tiki et al., 2016). However, this method is designed specifically for ABSA, utilizing predefined knowledge, and is only suitable for UDA setups where source domain labeled data is abundant. Our work presents a novel domain counterfactual generation algorithm, which can be trained in an unsupervised manner, and its generated outputs are demonstrated to be effective in multiple low-resource DA tasks. In this section, we formally define the concept of domain-counterfactual textual examples (D-CON s) and discuss the motivation behind them. Definition x (cid:48) is a domain-counterfactual example (D-CON ) of x if it is a coherent human-like text that is a result of intervening on the domain of x and changing it to another domain, while holding everything else equal. Particularly, we would like the task label of x (cid:48) and x to be identical. Formally, given an example ( x, y )  D and a destination domain D (cid:48) , the goal of D-CON generation is to generate x (cid:48)  PD (cid:48) ( X | Y = y ) such that x (cid:48) (cid:39) D (cid:48) x , where (cid:39) D (cid:48) is the domain counterfactual operator. In this work, given a labeled source example x we aim to generate coherent human-like D-CON s from the unlabeled domains (see 1). We propose a D-CON generation algorithm, DoCoGen , consisting of two components. The first involves masking domain specific terms of the given example, yielding M ( x ) . The second is a controllable generation model G which takes as input M ( x ) and a domain orientation vector v (cid:48) . This vector specifies the destination domain D (cid:48) , controlling the semantics of the generated D-CON . Formally: DoCoGen ( x, D (cid:48) ) = G ( M ( x ) , v (cid:48) ) (cid:39) D (cid:48) x Motivation The NLP community has recently become increasingly concerned with spurious correlations (Geirhos et al., 2020; Wang and Culotta, 2020; Gardner et al., 2021). In the case of DA, spurious correlations may be defined as correlations between X and Y which are relevant only to a specific domain or in a certain sample of labeled examples. Such correlations may make a predictor f : X  Y brittle to domain shifts. Using counterfactuals w.r.t. a specific variable allows us to both estimate its effect on our predictor (Feder et al., 2021b; Rosenberg et al., 2021) or alleviate its impact on it (Kaushik et al., 2021). We focus on the latter, automatically generating D-CON s by intervening on the domain variable D . Adding these D-CON s to the training set of a predictor should reduce its reliance on domain-specific information and spurious correlations. From a DA perspective, enriching the training data with D-CON s is motivated by pivot features (2), which are frequent in multiple domains and are prominent for the task. D-CON s preserve language patterns, such as pivots, which are frequent in multiple domains. Consider the middle example in Table 1, pivot words (such as excellent and important ) are preserved in the D-CON , while non-pivots ( intereseting and well-paced ) are replaced due to the domain intervention. Accordingly, a model trained on an example and its D-CON is directed to focus on pivots rather than on non-pivots, consequently generalizing better OOD. We propose a corrupt-and-reconstruct approach for generating D-CON s from given source domain examples (Figure 1). We next extend on these two steps, and describe our filtering mechanism used to disqualify low quality D-CON s. 4.1 Domain Corruption The first step of generating a D-CON is to mask domain specific terms. In order to mask an example x  D with a destination domain D (cid:48) , we first mask all uni-grams w with m ( w, D , D (cid:48) ) >  , where  is a hyperparameter and m is a masking score that is defined later in this section. Then, we mask all the remaining bi-grams (that do not contain a masked uni-gram) according to the same masking threshold  . This process is repeated up to tri-gram expressions. The final output of the corruption step is a masked example M ( x ) . In Figure 1, the masking scores of uni-grams and bi-grams appear above the input words. An n-gram is masked if and only if its score is above a  = 0 . 08 threshold and the scores of its grams are lower. For example, system is not masked although the bi-gram entertainment system has a score above the  threshold, since entertainment is masked and the score of system is lower than  . Masking Score Let w be an n-gram and D be a domain with n D unlabeled examples. We denote the number of examples from D that contain w by # w |D . By assuming that domains have equal prior probabilities and by using the Bayes' rule, the probability of D given w can be estimated by P ( D = D| W = w )  # w |D +  n D , where  is a smoothing hyperparameter. We define the affinity of w to D to be:  ( w, D ) = P ( D| w )  (cid:18) 1  H ( D | w ) log N (cid:19) where N is the number of unlabeled domains and H ( D | w ) is the entropy of D | w , which is upper 7730 The entertainment system failed twice but the crew reactivated it quick. bounded by log N . Notice that higher H ( D | w ) values indicate that w is not related to any specific domain. Finally, we set the masking score of an n-gram w with an origin domain D and a destination domain D (cid:48) as follows: m ( w, D , D (cid:48) ) =  ( w, D )   ( w, D (cid:48) ) Note that m ( w, D , D (cid:48) )  [  1 , 1] . It can be negative due to the right hand side's subtrahend, which aims to prevent masking n-grams that are related to the destination domain and should appear in the counterfactual, like system in Figure 1. The second step of DoCoGen is a reconstruction step that involves a generative model, based on an encoder-decoder T5 architecture (Raffel et al., 2020). Given a masked example M ( x ) and a destination domain D (cid:48) , we concatenate a domain orientation vector v (cid:48) that represents D (cid:48) with the masked input's embedding vectors. Then, the concatenated matrix is passed as an input to the encoder-decoder model for counterfactual generation, yielding x (cid:48) . We next describe the mechanism behind domain orientation vectors. Domain Orientation Vectors In addition to the T5 embedding matrix (T5 Embeddings in Figure 1), we equip our model with another learnable embedding matrix, containing K  N orientation vectors, such that each domain is represented by K different vectors (Orientation Embeddings in Figure 1). We initialize the orientation vectors with the T5 embedding vectors of the domain names and the top K  1 representing words of each domain. The top representing words of domain D are those which reach the highest score of: log (# w |D + 1)  ( w, D ) . We use K orientation vectors to allow us generate a heterogeneous set of D-CON s for a given destination domain (see examples in A). We note that although the orientation vectors are initialized with vectors from the T5 embedding matrix, they have a different role and thus are likely to converge to different values during the training process. Training In the spirit of low resource learning, we would like to train DoCoGen in an unsupervised manner, i.e., without access to manually generated D-CON s. Therefore, we use the unlabeled data of our unlabeled domains. For each example x , we provide the model with M ( x ) , the corrupted version of x , and v , the orientation vector of D , and with x as the gold output. The model hence learns to reconstruct x given M ( x ) and v . Notice that the origin and the destination domains are the same, i.e, D = D (cid:48) , and the masking score is m ( w, D , D ) = 0 . Hence, for masking purposes, we randomly choose  D (cid:54) = D and plug it as the destination domain in the masking score. We then choose an orientation v for D , by randomly sampling either the domain name or one of its representing words as long as it appears in x . Finally, since the orientation vector parameters are trained as part of the reconstruction objective, we establish the connection between the orientation vector and the semantics of the completed example. Hence, we expect that at inference time examples will be properly transformed into their D-CON s. 7731 Inference Given ( x, D , D (cid:48) ) , we first mask the example to get M ( x ) and select one orientation vector v (cid:48) that represents D (cid:48) . 2 Together, the tuple ( M ( x ) , v (cid:48) ) forms the input, and accordingly the model generates a D-CON x (cid:48) (cid:39) D (cid:48) x . To increase the likelihood that x (cid:48) originates from D (cid:48) , we restrict the model to generate only tokens of the original example or tokens that are related to D (cid:48) and meet the condition: max i  1 ,...,N m ( w, D (cid:48) , D i ) >  . In order to properly apply DoCoGen within a DA pipeline, we introduce a filtering mechanism that disqualifies low quality D-CON s generated by DoCoGen . Particularly, we train a classifier to predict the domain of the original, human-written unlabeled examples, and use it to remove D-CON s if their predicted domain is not the given destination domain. In addition, we disqualify D-CON s with less than four words or when the word overlap with the original example is lower than 25% . We name DoCoGen when equipped with this filtering mechanism F-DoCoGen . We next assess DoCoGen in terms of its generated D-CON s, ensuring they: (i) belong to the correct domain and label (1, 2), and (ii) are fluent (3, 4). To this end, we collected 20 original reviews, equally distributed among four domains (the A, D, E, and K domains, see 6). We then applied DoCoGen to generate 60 D-CON s, 3 for each of the original reviews (see 6 for the DoCoGen training setup). Finally, we trained the VAE model of Russo et al. (2020) on labeled data (all the labeled data of the A, D, E, and K domains) and applied it to generate five reviews from each of the above four domains, with the same number of positive and negative reviews as in the set of original reviews. We then conducted a crowd-sourcing experiment where five nearly native English speakers rated each example, considering the following evaluation measures: (1) Domain relevance ( D . REL ) whether the topic of the generated text is related to its destination domain; (2) Label preservation ( L . PRES ) what is the label of the generated example (and we report whether the answer was identical to the desired label); (3) Linguistic Acceptability ( ACCPT ) how logical and grammatical the example is (on a 1-5 scale); and (4) Word error rate ( WER ) what is 2 B.3 presents the % of masked tokens in our experiments. the minimum number of word substitutions, deletions, and insertions that have to be performed to make the example logical and grammatical. 3 Table 2 reports our results. DoCoGen achieves high ACCPT scores and low WER scores, significantly outperforming its VAE alternative, which is known to struggle with longer texts (Shen et al., 2019; Iqbal and Qureshi, 2020). Interestingly, DoCoGen achieves compatible results to the original reviews, indicating the high quality of its generated texts. Finally, in more than 90% of the cases DoCoGen manages to change the example domain to the desired domain, and in 80% it preserves the original example label. In comparison, only 88% of the original examples were annotated as their gold label. In this subsection we describe our tasks and datasets, as well as the two DA setups which are the focus of this work. A full description of the number of samples in each dataset is found in Table 6. Sentiment Classification We follow a large body of prior DA work, focusing on the task of binary sentiment classification. Specifically, our experiments include six different domains: the four legacy product review domains (Blitzer et al., 2007) Books (B), DVDs (D), Electronic items (E) and Kitchen appliances (K); the challenging airline review dataset (A) (Nguyen, 2015; Ziser and Reichart, 2018b); and the restaurant (R) domain obtained from the Yelp dataset challenge (Zhang et al., 2015). The focus of this work is on low resource DA, and thus we randomly sample 100 labeled examples to form the training set for the following domains: A, D, E, and K. As described in 2, we explore two DA setups, UDA and ADA. For UDA, where the model has 3 We actually asked the annotators to edit the example and then measured the number of edit operations. access to unlabeled target domain data, we experiment with 12 cross-domain setups, including the following domains: A, D, E, and K. For ADA, where unlabeled data from the target domain is not within reach, we experiment with a total of 8 setups, including B and R as target domains, and A, D, E, and K as source domains. Our reported accuracy scores are averaged across 25 different seeds and randomly sampled training and development sets. Multi-label intent prediction Our second task is multi-label intent prediction of utterances from information-seeking conversations. We use the multi-domain MANtIS dataset (Penha et al., 2019), consisting of diverse conversations from the question-answering Stack Exchange portal. The authors provide manually annotated user intent utterances, with eight possible intent labels, such as information request , potential answer and greetings . Since we focus on low resource scenarios, we use only the five most common labels, as the frequency of the other three labels is less than 5%, and in some domains they are completely missing. The MANtIS dataset consists of 14 domains: Apple (AP), DBA (DB), Electronics (EL), Physics (PH), Statistics (ST), askubuntu (UB); DIY (DI), English (EN), Gaming (GA), GIS (GI), Sci-Fi (SC), Security (SE), Travel (TR) and Worldbuilding (WO). We use the first 6 domains as unlabeled domains, randomly sampling train, development and test sets for each. The remaining 8 domains are used as target domains in the ADA setup, resulting in 30 UDA ( 6  5 ) and 48 ADA ( 6  8 ) setups. Following Penha et al. (2019), we use the (Macro) F1-score to measure classifier performances, and, like in the sentiment classification task, our reported results are averaged across 25 different seeds and randomly sampled training sets. DA by Augmentation The DA pipeline includes a T5-based sentiment classifier trained on labeled data from a single source domain and an augmentation model (e.g., DoCoGen ) trained on unlabeled data from four unlabeled domains. We first train DoCoGen on the unlabeled data, and then use it for generating D-CON s that enrich the classifier's training data. For each labeled training example, DoCoGen generates K = 4 D-CON s w.r.t. each unlabeled domain, resulting in a total of 16 D-CON s per example. After training the sentiment classifier on the enriched data, we evaluate it on test examples originating from one of the unlabeled domains (UDA) or one of the unseen domains (ADA). We denote each DA model by the algorithm that was used for enriching its training data. Our main models are DoCoGen and F-DoCoGen , which is equipped with the filtering mechanism. We compare them to three types of models: (a) baseline models, including both baselines for the entire DA pipeline (1,2,5) and alternative augmentation methods (3,4); (b) ablation models (6,7) that use variants of our D-CON generation algorithm where one component is modified, highlighting the importance of our design choices; and (c) an upper-bound generation model that has access to labeled data from the target domains. Unless otherwise stated, all sentiment classifiers use the same architecture, based on a pre-trained T5 model. We next describe the models in each of these groups. Baseline DA Models We experiment with five baselines: (1) No-Domain-Adaptation ( NoDA ), A model that is only trained on the available training data from the source domain in each DA setup; (2) Domain-Adversarial-Neural-Network ( DANN ), A model that integrates the sentiment analysis predictive task with an adversarial domain classifier to learn domain invariant representations (Ganin et al., 2016). This model does not apply augmentation, but instead the unlabeled data is used for training its adversarial component; (3) Easy-Data-Augmentation ( EDA ) , an augmentation method that randomly inserts, swaps, and deletes words or replaces synonyms (Wei and Zou, 2019); (4) Random-masking Random-Reconstructing ( RM-RR ), another basic augmentation method that randomly masks tokens from the input example and then fills the masks with tokens that are chosen by a masked language modeling head, as suggested by (Ng et al., 2020); and (5) PERL , a SOTA model for the UDA setup (Ben-David et al., 2020). Ablation Models We consider two variants of DoCoGen : (6) No-Orientation-Vectors ( No-OV ), a generation model that masks tokens by employing a similar masking mechanism as DoCoGen , and then employing a masked language modeling head to fill the masked tokens (without domain orientation vectors); and (7) Random-Masking with Orientation-Vectors ( RM-OV ), a generation model that randomly masks tokens from the input example and then employs the DoCoGen 's reconstruction mechanism to fill the masks. Upper-Bound We implement an upper-bound model for D-CON augmentation, Oracle-Matching Oracle-Gen ). Unlike all other models in this work, Oracle-Gen has access to target domain labeled data. Thus, given an example from a source domain, Oracle-Gen looks for the most similar example with the same label in the target domain, and adds it to its training data (see B.1). Tables 3 and 4 present sentiment classification accuracy results for the 12 UDA and 8 ADA setups, respectively. Table 5 presents the average intent prediction F1 scores for each source domain, taken across all target domains, in both UDA and ADA. D-CON Generation Impact For sentiment classification, our model, F-DoCoGen , outperforms all baseline models ( NoDA , DANN , EDA , and RM-RR ) in 10 of 12 UDA setups and in all ADA setups, exhibiting average performance gains of 1 . 9% and 1 . 3% over the best performing baseline model in the UDA ( DANN ) and the ADA ( RM-RR ) setups, respectively. Moreover, DoCoGen without filtering, is also superior to all baselines, reaching average gains of 1 . 6% and of 0 . 6% across all UDA and ADA setups, respectively. For intent prediction, DoCoGen (without filtering) is the best performing model, outperforming all baselines across all setups, and reaching average gains of 1 . 6% and 1 . 5% across all UDA and ADA setups, respectively. Since many intent examples are not domain-specific, our filtering mechanism tends to easily remove their DoCoGen generated D-CON s. We believe that this is the reason for the small degradation in F-DoCoGen performance compared to DoCoGen . However, F-DoCoGen still consistently outperforms all baselines. These results highlight the impact of D-CON generation on model robustness in low-resource setups. Finally, our models are also stable: Their std is lower than all baselines (see C.1). Ablation Models The tables further demonstrate that F-DoCoGen outperforms its ablation models ( 6.2), namely No-OV and RM-OV , in 10 of 12 and 7 of 8 UDA and ADA sentiment classification setups, respectively, and the same holds for DoCoGen across all intent prediction setups. Furthermore, in sentiment classification, F-DoCoGen achieves an average error reduction of 11 . 2% and 5 . 0% in UDA and ADA, respectively, over the strongest ablation model ( RM-OV ), while in intent prediction DoCoGen achieves a reduction of 8% and 7 . 6% , in both setups, respectively. Fi-7734 Source AP DB EL PH ST UB AVG Setup UDA ADA UDA ADA UDA ADA UDA ADA UDA ADA UDA ADA UDA ADA NoDA 75 . nally, our results demonstrate the importance of inappropriate D-CON s disqualification, as in the task of sentiment classification, F-DoCoGen outperforms DoCoGen in 8 of 12 UDA setups and in all ADA setups. On the other hand, when non domain-specific examples are frequent, filtering might lead to small performance degradation, as happens in the intent prediction task. Our results hence stress the importance of each of DoCoGen 's algorithmic components, i.e. domain-corruption ( 4.1 F-DoCoGen vs RM-OV ) and oriented-reconstruction ( 4.2 F-DoCoGen vs No-OV ). Complementary Effect with SOTA Models We notice that F-DoCoGen replicates the average performance of PERL (Ben-David et al., 2020), the UDA SOTA, in sentiment classification. However, since PERL is based on a different architecture than the rest of the models (BERT vs T5), the models are not directly comparable. PERL is a pivot-based representation learning method for DA, which applies pre-training on unlabeled target data and is hence relevant only for UDA. Since DoCoGen implements a different approach to DA (D-CON generation), we check for the complementary effect of these models: DoCoGen-PERL first augments the labeled data with D-CON s and then continues with the PERL pipeline. As reported in Table 3, DoCoGen-PERL outperforms PERL in 8 of 12 UDA setups, providing an average improvement of 0 . 7% . Furthermore, the average std of DoCoGen-PERL is 2 . 1 compared to 3 . 6 of PERL (C.1). This stresses the stability of DoCoGen-PERL across these challenging setup (Ziser and Reichart, 2019). Unfortunately, we cannot perform an equivalent comparison in the ADA setup, since its SOTA models (Ben-David et al., 2021; Wright and Augenstein, 2020) employ labeled data from multiple sources. Likewise, since PERL is not designed for multi-label prediction, we could not apply it to intent prediction. To the best of our knowledge, we are the first to effectively perform single-source ADA. Training Size Effect We would next like to understand the effect of D-CON s generated by DoCoGen on classifiers trained with manually labeled training sets of various sizes. Figure 2 shows that the effect of D-CON augmentation vanishes when the unaugmented classifier reaches accuracy above 85% and a performance plateau (visualized as an elbow in the curve). These results support our hypotheses that low-resource DA scenarios may result in a model that latch on spurious domain correlations, impeding its performance. Accordingly, generating D-CON s by intervening on the domain essentially reduces the reliance on domain-specific information and spurious correlations. We presented DoCoGen , a corrupt-and-reconstruct approach for generating domain-counterfactuals (D-CON s) and apply it as a data augmentation method in low-resource DA. We hypothesized that D-CON s may mitigate the reliance on domain-specific features and on spurious correlations and help generalize out of domain. Our augmentation strategy yields robust models that outperform strong baselines across many low-resource DA setups. In future work we would like to further improve the controllable generation quality of DoCoGen , potentially extending it to control for multiple attributes. Moreover, we would like our methodology to address additional NLP tasks and DA setups. We would like to thank the action editor and the reviewers, as well as the members of the IE@Technion NLP group for their valuable feedback and advice. This research was partially funded by an ISF personal grant No. 1625/18.
abstain objective method abstain result objective result abstain abstain abstain objective other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method method method method objective abstain objective abstain other other other other other other objective method method other other other other other other other other other other other objective abstain method other other other other other other other other method other other objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain method method abstain abstain method method method method abstain abstain abstain method abstain method abstain abstain method method method abstain method method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain method abstain method method method method method method method method abstain method other method abstain method abstain method abstain method method method method abstain method abstain abstain method method other method method method abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain method objective result objective abstain other other