Automatic transfer of text between domains has become popular in recent times. One of its aims is to preserve the semantic content of text being translated from source to target domain. However, it does not explicitly maintain other attributes between the source and translated text, for e.g., text length and descriptiveness. Maintaining constraints in transfer has several downstream applications, including data augmentation and de-biasing. We introduce a method for such constrained unsupervised text style transfer by introducing two complementary losses to the generative adversarial network (GAN) family of models. Unlike the competing losses used in GANs, we introduce cooperative losses where the discriminator and the generator cooperate and reduce the same loss. The first is a contrastive loss and the second is a classification loss  aiming to regularize the latent space further and bring similar sentences across domains closer together. We demonstrate that such training retains lexical, syntactic, and domain-specific constraints between domains for multiple benchmark datasets, including ones where more than one attribute change. We show that the complementary cooperative losses improve text quality, according to both automated and human evaluation measures. 1 1 Introduction Modern neural networks methods are capable of mapping data from one domain to another. Prominent examples include translation of text between languages (Vaswani et al., 2017; Artetxe et al., 2018; Lample et al., 2017), emoji creation from human faces (Taigman et al., 2017), and stylistic transfer of speech (Yuan et al., 2021). In Natural Language Processing (NLP), the umbrella term attribute transfer (Jin et al., 2020b) (or domain  The first two authors contributed equally 1 https://github.com/abhinavkashyap/dct I really loved Murakami's book Personal Pronoun Proper Noun Loved the movie Text Style Transfer Smaller Length No Personal Pronoun No Proper Noun I absolutely enjoyed Spielberg's direction Text Style Transfer + Constraints Similar Length Similar Personal Pronoun Domain appropriate Proper Noun vs. Figure 1: Illustrative example showing transfer of text from books to movies while maintaining constraints of identity. transfer ) refers to similar methods 2 . The aim is to maximally preserve the semantics of the source sentence (content) but change other properties (attributes), such as sentiment (Jin et al., 2020b), expertise (Cao et al., 2020), formality (Rao and Tetreault, 2018) or a combination of them (Subra-manian et al., 2018). Text style transfer , a popular form of attribute transfer, regards style as any attribute that changes between datasets (Jin et al., 2020a). Building on the progress of supervised transfer models, recent works have focused on unsupervised style transfer that avoids costly annotation of parallel sentences. However, models built using unsupervised methods perform poorly when compared to supervised (parallel) training (Artetxe et al., 2020). These methods, while capable of achieving the target domain characteristics, often fail to maintain the invariant content. Figure 1 illustrates one such example, where a sentence from the BOOKS domain is translated to the MOVIE domain. While the translated sentence  Loved the movie  has correctly transferred the attribute (style), it does not have the same length, does not retain the personal noun ( I ), nor use a domain-appropriate proper noun. Comparatively, the higher-fidelity transfer  I absolutely enjoyed Spielberg's direction , maintains such constraints of identity , in addition to being apt. While the literature primary utilizes the term style transfer , we adopt the more general term attribute as suggested by Jin et al. (2020a). of text transfer, as enforcing constraints of identity can help maintain the brand identity when the product descriptions are mapped from one commercial product to another. They can also help in data augmentation for downstream domain adaptation NLP applications ( 5). Constraints of identity are explored extensively in the computer vision task of cross-domain image generation. (Taigman et al., 2017), but these issuesto the best of our knowledgeare unexplored in NLP. In this paper, we improve unsupervised attribute transfer by enforcing invariances via explicit constraints. Current methods in text attribute transfer lack mechanisms to explicitly enforce such constraints between the source and the transferred sentence. To this end, we build upon unsupervised text style transfer work by introducing an additional explicit regularization component in the latent space of a GAN-based seq2seq network through two complementary losses ( 3). Unlike the adversarial losses in the GAN framework, our proposed losses cooperatively reduce the same objective. The first loss is a contrastive loss (Le-Khac et al., 2020) that brings sentences that have similar constraints closer and pushes sentences that are dissimilar farther away. The second loss is a classification loss that helps maintain the sentence identity via constraints from the latent vectors (Odena et al., 2017). Our approach, while simple and aimed at maintaining constraints, improves the overall performance of the generation. We demonstrate these gains over three datasets: YELP (Zhao et al., 2018b), IMDB (Dai et al., 2019) and POLITICAL (Prabhumoye et al., 2018), generating six constraints including lexical, syntactic and domain-specific constraints. The introduced cooperative losses satisfy the constraints more effectively compared against strong baselines. Since multiple attributes can change between two domains (Subra-manian et al., 2018), we test our method on one such dataset and show that the constraints of identity are maintained more effectively ( 4.4.2). To the best of our knowledge, our approach is the first to introduce cooperative losses in a GAN-like setup for NLG. Task Setup: We consider two sets of sentences (or corpora) S = { x 1 src , x 2 src , . . . x msrc } and T = { x 1 trg , x 2 trg , . . . x ntrg } , as the source and target domains, respectively. Each corpus  which we interpret as domains  contain discernable attributes, ranging from sentiment (e.g., positive vs. negative), topics, political slant (e.g., democratic vs. republi-can), or some combination (Li et al., 2018; Lample et al., 2019). The overall task is to rewrite a piece of text s i  S to t i  T , such that the translation changes the attributes varying across the two domains but retains the remaining content. While content retention is not explicitly defined in the literature, we design this new task of constrained unsupervised attribute transfer that assigns explicit constraints C = { c 1 , c 2 , . . . , c |C| } , to be retained. These constraints can be defined at various levels of a sentence: lexical, syntactic and domain-specific. Adversarially Regularized Autoencoder ( ARAE ): To perform unsupervised attribute transfer, we consider seq2seq models that encode source sentences to a latent space and then decodes them to the target sentences. ARAE s (Zhao et al., 2018b) are the auto-encoder variants of the Generative Adversarial Network (GAN) (Goodfellow et al., 2014) framework. They learn smooth latent spaces (by imposing implicit priors) to ease the sampling of latent sentences. ARAE s have been widely adopted in tasks like unsupervised text generation (Huang et al., 2020), topic modeling (Hu et al., 2020), among others, and form the backbone of our proposed model. ARAE consists of an auto-encoder with a deterministic encoder enc  : X  Z that encodes sentences into a latent space; i.e., z = enc  ( x )  P z , and a conditional decoder p  ( x | z ) that generates a sentence given a latent code. ARAE regularizes this latent space utilizing a GAN-like setup that includes an implicit prior obtained from a parameterized generator network enc  : N (0 , I )  Z . Here, enc  maps a noise sample s  N (0 , I ) to the corresponding prior latent code z = enc  ( s )  P  z . A critic crc  : Z  R then learns to distinguish between real and generated samples, whereas both enc  and enc  are adversarially trained to fool the critic. This results in a minimax optimization which implicitly minimizes the JS-Divergence between the two distributions P z and P  z : min  max  E z  P z [ crc  ( z )]  E z  P  z [ crc  ( z )] (1) The training involves three optimizations: i ) reducing the auto-encoder loss L ae , which tries to reconstruct the input and encourages copying be-417 havior and maintain semantics similar to the original text (Eq. 2); ii ) optimizing the critic's loss L cri to distinguish between real and fake samples (Eq. 3) iii ) training the encoder and generator L adv to fool the critic (Eq. 4): L ae ( ,  ) = E z  P z [  log p  ( x | z )] (2) L crc (  ) =  E z  P z [ crc  ( z )] + E z  P  z [ crc  ( z )] (3) L adv ( ,  ) = E z  P z [ crc  ( z )]  E z  P  z [ crc  ( z )] (4) 3 Proposed Method 3.1 Base Model ( ARAE seq2seq ) While ARAE is an auto-encoder that recreates input x  x , our requirement is to translate sentences from one domain to another. Given this, we modify the ARAE to a seq2seq variant such that we can translate input sentences between source and target domains; i.e., x src  x tgt and x tgt  x src . To achieve this, we utilize enc  to encode x src and repurpose enc  to encode x tgt . We obtain their latent codes ( z , z ) which we name as ( z s , z t ) , i.e., z s = enc  ( x src ) and z t = enc  ( x tgt ) . Next, to generate sentences, we consider two decoders x src  p  ( x | z ) and x tgt  p  ( x | z ) . Here, z can be either z s or z t based on whether we auto-encode (e.g., p  ( x | z s = enc  ( x src )) ) or translate (e.g., p  (cid:0) x | z t = enc  ( x tgt ) (cid:1) ). Unlike ARAE 's single decoder, we incorporate two decoders to enable bi-directional translation. In the above process, instead of sampling s from a noise distribution like N (0 , I ) and passing it through a generator enc  , we feed it text from the target domain T and a decoder dec  that decodes text in T . This is inspired from Cycle-GAN (Zhu et al., 2017), where instead of matching the noise distribution N , we match the distribution of T . In addition, we tie the weights of the encoders from both domains, so that the encoders learn to encode domain-agnostic information. Tying encoder weights has also been used by unsupervised machine translation (Artetxe et al., 2018; Lample et al., 2017) and multiple other works (Mai et al., 2020; Huang et al., 2020; Hu et al., 2020; Artetxe et al., 2018) 3 . While the latent space in ARAE seq2seq learns to match S and T sentences, there is no guarantee on translations maintaining the content. This issue is particularly pronounced in unsupervised attribute transfer due to lack of parallel sentences between S and T . To alleviate the issue, we propose to learn a structured latent space which embodies notions of our constraints in its embedded latent codes. This ensure that instances with similar constraints are closer in the latent space. In particular, we propose two types of optimization  self-supervised and discriminative  to maintain the constraints better. We use contrastive representation learning to regularize the latent space, such that encoders bring two sentences sharing similar constraints closer together (positive pairs), and force dissimilar ones away (negative pairs). For example, sentences of similar lengths (irrespective of their domains) should be closer together. Among many self-supervised metric losses such as Triplet Loss (Hoffer and Ailon, 2015) and NT-3 We tried with separate encoders and decoders, but encoders with tied weights work best 418 Algorithm 1: ARAE seq2seq + CLF + CONTRA 1 for each training iteration do 2 1) Train the Auto-encoders: 3 Sample x src  S , x trg  T 4 z s = enc  ( x src ) , z t = enc  ( x trg ) 5 Backprop loss, L ae ( ,  ) , L ae ( ,  ) 6 2) Train the Critic: 7 Sample x src  S , x trg  T 8 z s = enc  ( x src ) , z t = enc  ( x trg ) 9 z scrc = crc hid ( z s ) , z tcrc = crc hid ( z t ) 10 l crc  L crc (  ) 11 2a) Critic Co-op Training: 12 Backprop loss, l crc +  1 L con (  ) +  2 L clf ( ,  ) 13 3) Adversarial Training: 14 Sample x src  S , x trg  T 15 z s = enc  ( x src ) , z t = enc  ( x trg ) 16 Backprop loss, L adv ( ,  ) 17 3a) Encoder Co-op Training: 18 Backprop loss,  1 L con ( ,  ) +  2 L clf ( , ,  ) Xent loss (Chen et al., 2020), we use one that is amenable to multiple positive instances (Khosla et al., 2020). Given a sentence s i  S in a mini-batch of size B , we mine P positive sentences each from S and T that share the same constraints with s i . This contrastive loss is given by: L con ( , ,  ) =  1 | P | log (cid:32) P (cid:88) j =1 e ( z i  z j ) (cid:80) B \{ i } k =1 e ( z i  z k ) (cid:33) (5) where z 's are representations obtained from the encoders in S , T or representations obtained from the last layer of critic crc  . C i are a set of constraints for a sentence. Recently, (Kang and Park, 2020) introduced the cooperative loss in the adversarial setup where contrastive losses are added to both the critic and generator for GANs. Unlike the normal opposing losses of the generator and the critic, both of them cooperatively reduce the contrastive loss. We follow a similar principle and add the loss to both the encoders and the critic (Lines 18). Contrastive learning might be sub-optimal if we do not mine good quality positive and negative samples (Tian et al., 2020). To address this, we propose another way to regularize the latent space. Similar to ACGAN (Odena et al., 2017), we encourage the encoders and the critic to cooperatively reduce a classification loss. We include a classifier D  : Z  R |C| that predicts the different constraints C of the sentences and the binary cross entropy loss is reduced. where |C| is the number of constraints per sentence,  is the sigmoid function and l c are the logits produced by the classifier for z i . As in contrastive loss, the z i can be produced by encoders of S , T or from the hidden layers of the critic. Datasets. We use three datasets with single attribute changes: i ) Yelp Reviews : business reviews listed on Yelp, labeled as either a positive or negative sentiment. ii ) IMDb Movie Reviews : consists of movie reviews (Dai et al., 2019) also labelled as positive or negative. iii ) Political Slant : consists of Facebook posts from the politicians of the United States Senate and the House of Representatives (Prabhumoye et al., 2018), labeled with either democratic/republican slant. We provide a summary of the dataset statistics in Table 1. We include datasets of varied length and complexity. Apart from having different topics, the IMDB dataset is more formal compared to the more colloquial YELP . We fix the maximum vocabulary size for YELP , IMDB and POLITICAL at 30K which is also the default maximum vocab size used in (Zhao et al., 2018b). ii ) Syntactic : Presence of personal pronouns (binarized to indicate the presence of a personal pronoun ); number of adjectives (categori-cal up to 5); number of proper nouns (categorical up to 3); syntactic tree height (categorical up to 10). iii ) Domain specific  number of domain-specific attributes (Li et al., 2018) (categorical up to 5). Further, we label the sentence with a constraint-specific, catch-all label if the bounds are beyond what we mention above. Since the distribution of the labels may be different, we report the F1 score on our constraints. For the encoders, we use a one-layer LSTM network with 300 hidden dimensions for all the datasets. For the critics and classification loss, we use a two-layer multi-layer perceptron with 100 hidden units. Training Hyper-parameters: For all our experiments we set the learning rate of the auto-encoder ( lr ae ) to 1e-3 and ( lr disc ) to 1e-4. The number of discriminator steps ( n dis ) is set to 5. The Adam optimizer parameters  1 = 0 . 5 and  2 = 0 . 9 , which ensures a more conservative optimization and is known to improve stability. We also add a gradient penalty to the loss function of the discriminator that stabilizes training. All the suggestions for stabilizing training are mostly obtained from (Arjovsky and Bottou, 2017). Automatic Evaluation: Our automatic evaluation considers the following three prominent criteria: i ) Semantic Similarity ( SIM ): Measured between source and translated target sentences using encoders (Wieting et al., 2019), instead of n-gram metrics like BLEU (Papineni et al., 2002) which have weak correlations with human judgments. ii ) Transfer Accuracy ( ACC ): The transferred sentence should belong to the target domain and a classifier is trained to distinguish between the source and the target sentence. We use fastText classifiers (Joulin et al., 2017) for every dataset. We achieve accuracy of 97 . 9 for YELP , 96 . 9 for IMDB and 97 . 1 for POLITICAL . iii ) Fluency ( FL ): A transferred sentence should be grammatically correct. We fine-tune a RoBERTa-large model on the COLA (Warstadt et al., 2018) dataset to indicate whether a sentence is linguistically acceptable. Finally, we combine the three scores into an aggregate, following the criteria suggested by Krishna et al. (2020): AGG = 1 | S | (cid:88) s  SACC (s)  SIM (s)  FL (s) Human Evaluation: We also perform an indicative human evaluation where we randomly sample 100 samples from each of the three datasets and hire three researchers to rate every sentence for FL , SIM and ACC on a 3-point scale (Krishna et al., 2020). We compare ARAE seq2seq with the following baselines: a) DRG : The Delete, Retrieve, Generate method that deletes domain specific attributes, retrieves a template and generates the target domain text (Li et al., 2018). We use the stronger, entire system rather than the weaker DELETEONLY and RETRIEVEONLY baselines; b) ARAE : Adversarially regularized autoencoders our system is based on (Zhao et al., 2018b); c) ARAE seq2seq : Our model without the contrastive learning or cooperative classifier; d) ARAE seq2seq + CONTRA : Our model with the contrastive learning; e) ARAE seq2seq + CLF : Our model with the cooperative classifier; f) ARAE seq2seq + CLF + CONTRA : Our model with both the cooperative losses. The closest model to ours is from (Huang et al., 2020). However, we were not able to reproduce the results. 4 4.4 Results 4.4.1 Overall Results ARAE seq2seq + CONTRA and ARAE seq2seq + CLF consistently perform better than DRG and ARAE on the AGG score (Table 2). The AGG for YELP is 20.6 (vs. 19.8), for IMDB it is 28.1 (vs. 19.9) and for POLITICAL 25.5 (vs. 11.0). Although cooperative loss reduction aims to satisfy the constraints between two domains, our results show that further regularization of the latent space not only brings advantages in satisfying the constraints but also improves performance (Lavoie-Marchildon et al., 4 Repeated attempts to obtain the original source code failed. 2020). Effect of Cooperative Loss Reduction on ACC FL and SIM : Across datasets, reducing cooperative losses improves ACC and FL and SIM to ARAE . Although DRG produces sentences with high SIM as most of the text from the original sentence is retained after the delete step, there is a large tradeoff with ACC resulting in low AGG scores. Also, compared to ARAE , adding cooperative losses significantly increases the SIM , with the highest increase observed for POLITICAL . The reasons for this could be two-fold: i ) since we mine positive sentences from a corpus that is grounded in real world events, most lexically-similar sentences may also be semantically similar (Guu et al., 2018), and ii ) since we tie the encoders from the source and target domain, we extract domain-agnostic information before generation, which retains content. Fluency ( FL ) also improves over all datasets. We hypothesize that reducing cooperative losses regularizes the latent space bringing fluent sentences closer together, enabling the decoder to produce semantically similar and linguistically acceptable sentences. The improvement for POLITICAL is less; we find these source sentences themselves are less fluent and contain many U.S. political acronyms, and that our system produces many out-of-vocabulary words affecting fluency. Nucleus Sampling: Our system achieves the highest AGG score with greedy decoding. We also experiment with nucleus sampling (Holtzman et al., 2019) with different p values. We report results for only p = 0 . 6 in Table 2, as it produced the best result. With p =0.6, the results are more diverse, increasing ACC as expected. However we find that with higher values of p , there is a trade-off with SIM resulting in a lower AGG score overall  similar to Krishna et al. (2020). Effect of the Number of Positives: The number of positive and negative samples used for contrastive learning (Eq. 5) have a significant effect on the overall performance (Khosla et al., 2020; Chen et al., 2020; Henaff, 2020). Table 3 ( rows | P |  { 1 , 2 , 5 , 10 } ) shows the AGG scores on IMDB (for one of the runs), for different number of positives. We find that AGG is the highest with 2 positives per sample as also used by Khosla et al. (2020). Although increasing the number of negatives is beneficial for contrastive learning, when more than one positive example is available, using them brings further improvements (Khosla et al., 2020). Cooperative Losses are Important on Both the Generator and Critic: Table 3 shows the importance of adding the cooperative losses on the generator and critic. First, we see that adding the cooperative losses on both the generator and the critic is crucial for the overall performance. While adding the cooperative contrastive loss to both the generator and critic increases FL and ACC while maintaining similar levels of SIM , adding the cooperative classification loss improves SIM which shows the complementary nature of the losses. Human Evaluation: We average the results and present it in Table 4. DRG produces marginally better semantically similar sentences. Compared to ARAE , our model performs well except for in YELP . This may be because we use nucleus sampling with 0.9 which optimizes for diversity rather than similarity. On other metrics we perform on par or better than our competing systems. (See Appendix B) Qualitative Examples: Table 5 shows examples of the quality of transferred examples (see Appendix A for more). Mistakes made by the model can be attributed to poor understanding of the original semantics, lack of diversity, and not producing attribute-specific words. Figure 3 shows that introducing the cooperative losses significantly outperform DRG and ARAE in maintaining constraints. Specifically the ARAE seq2seq + CLF model performs better than ARAE seq2seq + CONTRA . One reason could be that, finding the appropriate positives and strong negatives can be problematic for contrastive learning. On the other hand, the classifier's objective is simpler and forces the encoder to produce representations that satisfy the different constraints effectively. A seemingly easy to maintain constraint is the length of the sentence. However, seq2seq systems have a difficulty of maintaining appropriate lengths (Murray and Chiang, 2018). With no additional regularization ARAE does not maintain the length as well as ARAE seq2seq + CLF . On the other hand, compared to the lexical constraints, syntactic attributes like descriptiveness, tree height and domain specific constraints present challenges, with significantly lower F scores. ARAE seq2seq + CLF produces significantly better results in maintaining them. This shows that obtaining improvements on the overall AGG does not necessarily translate to producing outputs that satisfy constraints. DRG maintains the proper noun for IMDB effectively, because it contains a wide variety of actor and movie names. They are retained verbatim after the delete operation. Multiple Attribute Datasets: To test whether our model can satisfy constraints across domains where multiple attributes change, we use the multi-attribute dataset released by (Lample et al., 2019). We chose the ASIAN and MEXICAN as two domains. Each of these domains can have multiple attributes like positive and negative sentiment text, different gender attributions to sentences, etc. We compare our ARAE seq2seq + CLF model with the ARAE seq2seq and ARAE in Figure 4. The results are more pronounced in this case with ARAE seq2seq + CLF having clear advantage over ARAE seq2seq . This shows that even with multiple attributes changing between domains, cooperatively reducing losses can satisfy different constraints more effectively. Qualitative Examples: Table 6 shows examples of our model maintaining constraints compared to ARAE . Sometimes, ARAE hallucinates and adds personal pronouns like my to the text even when there are no personal pronouns ( row 1). Also, our model produces sentences where the number of proper nouns are retained (Chris Klein vs. Robert De Niro), whereas ARAE does not. Cycle Consistency Loss: a) In Latent Spaces Cycle consistency in latent spaces has been shown to improve word-level tasks, such as cross-lingual dictionary construction (Mohiuddin and Joty, 2019) and topic modeling (Hu et al., 2020). A recent work from (Huang et al., 2020) claims to improve unsupervised style transfer using such losses. In our experiments, however, it did not result in any noticeable performance improvement 5 . Given this, we hypothesize that cycle consistency might be too restrictive for sentence-level tasks. b) Using Back-Translation Back-translation is another alternative to ensure semantic consistency between source and the target sentence (Prabhumoye et al., 2018; Artetxe et al., 2018; Lample et al., 2017). However, in our case, since we are training an ARAE , it would involve an additional inference and auto-encoder training step which is expensive and we defer exploring this. Using Transformers: We also replace our LSTM auto-encoders with both pre-trained and randomly initialized transformer encoderdecoders (Rothe et al., 2020). Although we found an increase in the AGG , it was mostly because of very high SIM and very low ACC . Reducing the number of layers, attention heads would still result in a large model that is still prone to copying text. This reveals the potential limitations of our method and training using transformers is a future work. Transferred sentences as Adversarial Examples: We demonstrate an important application of our proposed constrained transfer by considering them as adversarial examples for domain adaptation. Domain Adversarial Neural Network (DANN) (Ganin et al., 2017) is an unsupervised domain adaptation method that improves performance of an end-task (e.g, sentiment analysis) on a target domain considering only supervised data from source domain. We train DANN for sentiment analysis on amazon reviews dataset (He and McAuley, 2016) with DVD as source and ELECTRONICS as the tar-5 Repeated attempts to obtain source codes failed. Next, we train the best variant of ARAE seq2seq to transfer a separate set DVD reviews to ELECTRONICS reviews and use them as adversarial examples to test the DANN model 6 . We find that the accuracy of DANN on the ELECTRONICS domain reduces by  3 points. This shows the potential application of domain transferred sentences as adversarial examples. Similar ideas have been tried for image style transfer (Xu et al., 2020), but needs more investigation in NLP. Text attribute transfer has a vast literature (Jin et al., 2020a) with deep learning methods becoming popular. The methods are either supervised (requiring parallel data) or unsupervised. Supervised methods re-purpose Sequence to Sequence models used in machine translation to achieve the goals (Rao and Tetreault, 2018). However, obtaining parallel data is cumbersome and thus unsupervised methods that consider pseudo-parallel data have become popular. Disentanglement approaches are the prevalent approach to tackle unsupervised attribute transfer: attributes and content are separated in latent dimension. To disentangle the attributes adversarial methods maximize the loss of a pre-trained attribute classifier (Li et al., 2020; Fu et al., 2018; Zhao et al., 2018a; John et al., 2019). However, the literature has paid little attention in defining and preserving content. Cycle consistency losses  imposing that reconstruction from the target style sentence should resemble the source sentence  is the most prevalent (Prabhumoye et al., 2018; Logeswaran et al., 2018; Dai et al., 2019; Huang et al., 2020; Yi et al., 2020). However, this is expensive and nondifferentiable, thus requiring reinforcement learning techniques to enforce it. Our work defines the different constraints that should be preserved and adds simple differentiable contrastive learning losses to preserve them. In recent times, text style transfer models are moving away from disentanglement approaches (Subramanian et al., 2018). Recent works that use transformers for style transfer also have adopted this (Dai et al., 2019; Krishna et al., 2020). How-6 Since each of DVD and ELECTRONICS contain positive and negative reviews, we test whether transferred sentences maintain the appropriate sentiment and find the accuracy to be 79%. ever, these methods do not explicitly maintain the constraints between the two styles which is the main aim of our work. Text style transfer works focuses on retaining content and changing the style of sentences but does not maintain other desirable constraints. We address this by introducing two cooperative losses to the GAN-inspired Adversarially Regularized Autoencoder (ARAE) that further regularizes the latent space. While satisfying the constraints our methods brings significant improvements in overall score. While we focus on simple constraints at the sentenceand word-level, future work can add phrase-level and more fine-grained constraints. Potential future work may explore reinforcement learning losses to directly optimize the constraints. We would like to thank the anonymous reviewers for their useful suggestions. We would also like to acknowledge the support of the NExT research grant funds, supported by the National Research Foundation, Prime Ministers Office, Singapore under its IRC@ SG Funding Initiative, and to gratefully acknowledge the support of NVIDIA Corporation with the donation of the GeForce GTX Titan XGPU used in this research. The work is also supported by the project no. T2MOE2008 titled CSK-NLP: Leveraging Commonsense Knowledge for NLP awarded by Singapore's Ministry of Education under its Tier-2 grant scheme.
abstain abstain abstain abstain abstain method abstain objective result abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain result abstain method objective abstain abstain objective abstain abstain result objective method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other other method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain other abstain abstain abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain result abstain abstain result abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain method result abstain result abstain abstain result method result abstain abstain other other other other other other other other other method other other abstain objective abstain method result method abstain other other other other