We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., changing screen is just the right size to screen is too small ). Our training data includes only sentences labeled with their attribute (e.g., positive or negative), but not pairs of sentences that differ only in their attributes, so we must learn to disentangle attributes from attribute-independent content in an unsupervised way. Previous work using adversarial methods has struggled to produce high-quality outputs. In this paper, we propose simpler methods motivated by the observation that text attributes are often marked by distinctive phrases (e.g., too small ). Our strongest method extracts content words by deleting phrases associated with the sentence's original attribute value, retrieves new phrases associated with the target attribute, and uses a neural model to fluently combine these into a final output. On human evaluation, our best method generates grammatical and appropriate responses on 22% more inputs than the best previous system, averaged over three attribute transfer datasets: altering sentiment of reviews on Yelp, altering sentiment of reviews on Amazon, and altering image captions to be more romantic or humorous. The success of natural language generation (NLG) systems depends on their ability to carefully control not only the topic of produced utterances, but also attributes such as sentiment and style. The desire for more sophisticated, controllable NLG has led to increased interest in text attribute transfer the task of editing a sentence to alter specific attributes, such as style, sentiment, and tense (Hu  Work done while the author was a visiting researcher at Stanford University. et al., 2017; Shen et al., 2017; Fu et al., 2018). In each of these cases, the goal is to convert a sentence with one attribute (e.g., negative sentiment) to one with a different attribute (e.g., positive sen-timent), while preserving all attribute-independent content 1 (e.g., what properties of a restaurant are being discussed). Typically, aligned sentences with the same content but different attributes are not available; systems must learn to disentangle attributes and content given only unaligned sentences labeled with attributes. Previous work has attempted to use adversarial 1 Henceforth, we refer to attribute-independent content as simply content , for simplicity. networks (Shen et al., 2017; Fu et al., 2018) for this task, butas we demonstratetheir outputs tend to be low-quality, as judged by human raters. These models are also difficult to train (Salimans et al., 2016; Arjovsky and Bottou, 2017; Bous-malis et al., 2017). In this work, we propose a set of simpler, easier-to-train systems that leverage an important observation: attribute transfer can often be accomplished by changing a few attribute markers  words or phrases in the sentence that are indicative of a particular attributewhile leaving the rest of the sentence largely unchanged. Figure 1 shows an example in which the sentiment of a sentence can be altered by changing a few sentiment-specific phrases but keeping other words fixed. With this intuition, we first propose a simple baseline that already outperforms prior adversarial approaches. Consider a sentiment transfer (nega-tive to positive) task. First, from unaligned corpora of positive and negative sentences, we identify attribute markers by finding phrases that occur much more often within sentences of one attribute than the other (e.g., worst and very disppointed are negative markers). Second, given a sentence, we delete any negative markers in it, and regard the remaining words as its content. Third, we retrieve a sentence with similar content from the positive corpus. We further improve upon this baseline by incorporating a neural generative model, as shown in Figure 1. Our neural system extracts content words in the same way as our baseline, then generates the final output with an RNN decoder that conditions on the extracted content and the target attribute. This approach has significant benefits at training time, compared to adversarial networks: having already separated content and attribute, we simply train our neural model to reconstruct sentences in the training data as an auto-encoder. We test our methods on three text attribute transfer datasets: altering sentiment of Yelp reviews, altering sentiment of Amazon reviews, and altering image captions to be more romantic or humorous. Averaged across these three datasets, our simple baseline generated grammatical sentences with appropriate content and attribute 23% of the time, according to human raters; in contrast, the best adversarial method achieved only 12% . Our best neural system in turn outperformed our baseline, achieving an average success rate of 34% . Our code and data, including newly collected human reference outputs for the Yelp and Amazon domains, can be found at https://github.com/lijuncen/ Sentiment-and-Style-Transfer . We assume access to a corpus of labeled sentences D = { ( x 1 , v 1 ) , . . . , ( x m , v m ) } , where x i is a sentence and v i  V , the set of possible attributes (e.g., for sentiment, V = { positive , negative } ). We define D v = { x : ( x, v )  D} , the set of sentences in the corpus with attribute v . Crucially, we do not assume access to a parallel corpus that pairs sentences with different attributes and the same content. Our goal is to learn a model that takes as input ( x, v tgt ) where x is a sentence exhibiting source (original) attribute v src , and v tgt is the target attribute, and outputs a sentence y that retains the content of x while exhibiting v tgt . As a motivating example, suppose we wanted to change the sentiment of The chicken was deli-cious. from positive to negative. Here the word delicious is the only sentiment-bearing word, so we just need to replace it with an appropriate negative sentiment word. More generally, we find that the attribute is often localized to a small fraction of the words, an inductive bias not captured by previous work. How do we know which negative sentiment word to insert? The key observation is that the remaining content words provide strong cues: given The chicken was . . .  , one can infer that a taste-related word like bland fits, but a word like rude does not, even though both have negative sentiment. In other words, while the deleted sentiment words do contain non-sentiment information too, this information can often be recovered using the other content words. In the rest of this section, we describe our four systems: two baselines (RETRIEVEONLY and TEMPLATEBASED ) and two neural models (DELETEONLY and DELETEANDRETRIEVE ). An overview of all four systems is shown in Figure 2. Formally, the main components of these systems are as follows: 1. Delete : All 4 systems use the same procedure to separate the words in x into a set of 1866 (3) Generate output sentence Inputs (2) Retrieve similar sentence with target attribute RetrieveOnly i have had it for a while but barely used it . attribute markers a ( x, v src ) and a sequence of content words c ( x, v src ) . 2. Retrieve : 3 of the 4 systems look through the corpus and retrieve a sentence x tgt that has the target attribute v tgt and whose content is similar to that of x . 3. Generate : Given the content c ( x, v src ) , target attribute v tgt , and (optionally) the retrieved sentence x tgt , each system generates y , either in a rule-based fashion or with a neural sequence-to-sequence model. We describe each component in detail below. We propose a simple method to delete attribute markers ( n -grams) that have the most discriminative power. Formally, for any v  V , we define the salience of an n -gram u with respect to v by its (smoothed) relative frequency in D v : s ( u, v ) = count( u, D v ) +  (cid:16)P v 0 V ,v 0 6 = v count( u, D v 0 ) (cid:17) + , (1) where count( u, D v ) denotes the number of times an n -gram u appears in D v , and  is the smoothing parameter. We declare u to be an attribute marker for v if s ( u, v ) is larger than a specified threshold  . The attributed markers can be viewed as discriminative features for a Naive Bayes classifier. We define a ( x, v src ) to be the set of all source attribute markers in x , and define c ( x, v src ) as the sequence of words after deleting all markers in a ( x, v src ) from x . For example, for The chicken was delicious, we would delete delicious and consider The chicken was. . .  to be the content (Figure 2, Step 1). To decide what words to insert into c ( x, v src ) , one useful strategy is to look at similar sentences with the target attribute. For example, negative sentences that use phrases similar to The chicken was. . .  are more likely to contain bland than rude. Therefore, we retrieve sentences of similar content and use target attribute markers in them for insertion. Formally, we retrieve x tgt according to: x tgt = argmin x 0 D v tgt d ( c ( x, v src ) , c ( x 0 , v tgt )) , (2) where d may be any distance metric comparing two sequences of words. We experiment with two options: (i) TF-IDF weighted word overlap and (ii) Euclidean distance using the content embeddings in Section 3.3 (Figure 2, Step 2). Finally, we describe how each system generates y (Figure 2, Step 3). RETRIEVEONLY returns the retrieved sentence x tgt verbatim. This is guaranteed to produce a grammatical sentence with the target attribute, but its content might not be similar to x . TEMPLATEBASED replaces the attribute markers deleted from the source sentence a ( x, v src ) with those of the target sentence a ( x tgt , v tgt ) . 2 This strategy relies on the assumption that if two attribute markers appear in similar contexts , they are roughly syntactically exchangeable. For example, love and don't like appear in similar contexts (e.g., i love this place. and i don't like this place. ), and exchanging them is syntactically valid. However, this naive swapping of attribute markers can result in ungrammatical outputs. DELETEONLY first embeds the content c ( x, v src ) into a vector using an RNN. It then concatenates the final hidden state with a learned embedding for v tgt , and feeds this into an RNN decoder to generate y . The decoder attempts to produce words indicative of the source content and target attribute, while remaining fluent. DELETEANDRETRIEVE is similar to DELETEONLY , but uses the attribute markers of the retrieved sentence x tgt rather than the target attribute v tgt . Like DELETEONLY , it encodes c ( x, v src ) with an RNN. It then encodes the sequence of attribute markers a ( x tgt , v tgt ) with another RNN. The RNN decoder uses the concatenation of this vector and the content embedding to generate y . DELETEANDRETRIEVE combines the advantages of TEMPLATEBASED and DELETEONLY . Unlike TEMPLATEBASED , DELETEANDRETRIEVE can pick a better place to insert the given attribute markers, and can add or remove function words to ensure grammaticality. Compared to DELETEONLY , DELETEANDRETRIEVE has a stronger inductive bias towards using target attribute markers that are likely to fit in the current context. Guu et al. (2018) showed that retrieval strategies like ours can help neural generative models. Finally, DELETEANDRETRIEVE gives us finer control over the output; for example, we can control the degree of sentiment by deciding whether to add good or fantastic based on the retrieved sentence x tgt . We now describe how to train DELETEANDRETRIEVE and DELETEONLY . Recall that at training time, we do not have access to ground truth outputs that express the target attribute. Instead, we train DELETEONLY to reconstruct the sentences in the training corpus given their content and original attribute value by maximizing: L (  ) = X ( x,v src ) D log p ( x | c ( x, v src ) , v src );  ) . (3) For DELETEANDRETRIEVE , we could similarly learn an auto-encoder that reconstructs x from c ( x, v src ) and a ( x, v src ) . However, this results in a trivial solution: because a ( x, v src ) and c ( x, v src ) were known to come from the same sentence, the model merely learns to stitch the two sequences together without any smoothing. Such a model would fare poorly at test time, when we may need to alter some words to fluently combine a ( x tgt , v tgt ) with c ( x, v src ) . To address this train/test mismatch, we adopt a denoising method similar to the denoising auto-encoder (Vincent et al., 2008). During training, we apply some noise to a ( x, v src ) by randomly altering each attribute marker in it independently with probability 0 . 1 . Specifically, we replace an attribute marker with another randomly selected attribute marker of the same attribute and word-level edit distance 1 if such a noising marker exists, e.g., was very rude to very rude , which produces a 0 ( x, v src ) . Therefore, the training objective for DELETEANDRETRIEVE is to maximize: L (  ) = X ( x,v src ) D log p ( x | c ( x, v src ) , a 0 ( x, v src );  ) . (4) 4 Experiments We evaluated our approach on three domains: flip-ping sentiment of Yelp reviews (YELP ) and Amazon reviews (AMAZON ), and changing image captions to be romantic or humorous (CAPTIONS ). We compared our four systems to human references and three previously published adversarial approaches. As judged by human raters, both of our two baselines outperform all three adversarial methods. Moreover, DELETEANDRETRIEVE outperforms all other automatic approaches. First, we describe the three datasets we use, which are commonly used in prior works too. All datasets are randomly split into train, development, and test sets (Table 1). AMAZON Similar to YELP , each example is a sentence from a product review on Amazon, and is labeled as having either positive or negative sentiment (He and McAuley, 2016). CAPTIONS In the CAPTIONS dataset (Gan et al., 2017), each example is a sentence that describes an image, and is labeled as either factual, romantic, or humorous. We focus on the task of converting factual sentences into romantic and humorous ones. Unlike YELP and AMAZON , CAPTIONS is actually an aligned corpusit contains captions for the same image in different styles. Our systems do not use these alignments, but we use them as gold references for evaluation. CAPTIONS is also unique in that we reconstruct romantic and humorous sentences during training, whereas at test time we are given factual captions. We assume these factual captions carry only content, and therefore do not look for and delete factual attribute markers; The model essentially only inserts romantic or humorous attribute markers as appropriate.
method objective abstain objective objective method abstain abstain abstain abstain abstain method abstain abstain objective abstain objective abstain objective abstain method result result abstain objective result other method method method abstain method method method method abstain abstain method abstain abstain abstain abstain method method method method abstain method method abstain abstain method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method method abstain method method method abstain method abstain method method method abstain abstain abstain abstain abstain method abstain method abstain method