Understanding contrastive opinions is a key component of argument generation. Central to an argument is the claim, a statement that is in dispute. Generating a counter-argument then requires generating a response in contrast to the main claim of the original argument. To generate contrastive claims, we create a corpus of Reddit comment pairs self-labeled by posters using the acronym FTFY (fixed that for you). We then train neural models on these pairs to edit the original claim and produce a new claim with a different view. We demonstrate significant improvement over a sequence-to-sequence baseline in BLEU score and a human evaluation for fluency, coherence, and contrast. In the Toulmin model (1958), often used in computational argumentation research, the center of the argument is the claim, a statement that is in dispute (Govier, 2010). In recent years, there has been increased interest in argument generation (Bilu and Slonim, 2016; Hua and Wang, 2018; Le et al., 2018). Given an argument, a system that generates counter-arguments would need to 1) identify the claims to refute, 2) generate a new claim with a different view, and 3) find supporting evidence for the new claim. We focus on this second task, which requires an understanding of contrast. A system that can generate claims with different views is a step closer to understanding and generating arguments (Apothloz et al., 1993). We build on previous work in automated claim generation (Bilu et al., 2015) which examined generating opposing claims via explicit negation. However, researchers also noted that not every claim has an exact opposite. Consider a claim from Reddit: Get employers out of the business, pass universal single-payer healthcare . This is an example of a policy claim a view on what should be done (Schiappa and Nordin, 2013). While negation of this claim is a plausible response (e.g. asserting there should be no change by stating Do not get employers out of the business, do not pass universal healthcare ), negation limits the diversity of responses that can lead to a productive dialogue. Instead, consider a response that provides an alternative suggestion: Get employers out of the business, deregulate and allow cross-state competition . (2) In Example 1, the speaker believes in an increased role for government while in Example 2, the speaker believes in a decreased one. As these views are on different sides of the political spectrum, it is unlikely that a single speaker would utter both claims. In related work, de Marneffe et al. (2008) define two sentences as contradictory when they are extremely unlikely to be true simultaneously. We thus define a contrastive claim as one that is likely to be contradictory if made by the speaker of the original claim . Our goal, then, is to develop a method for generating contrastive claims when explicit negation is not the best option. Generating claims in this way also has the benefit of providing new content that can be used for retrieving or generating supporting evidence. In order to make progress towards generating contrastive responses, we need large, high-quality datasets that illustrate this phenomenon. We construct a dataset of 1,083,520 contrastive comment pairs drawn from Reddit using a predictive model to filter out non-contrastive claims. Each pair contains very similar, partially aligned text but the responder has significantly modified the original post. We use this dataset to model differences in views and generate a new claim given an original comment. The similarity within these pairs allows us to use them as distantly labeled contrastive word alignments. The word alignments provide semantic information about which words and phrases can be substituted in context in a coherent, meaningful way. 1. Methods and data for contrastive claim identification to mine comment pairs from Reddit, resulting in a large, continuously growing dataset of 1,083,520 distant-labeled examples. 2. A crowd-labeled set of 2,625 comments each paired with 5 new contrastive responses generated by additional annotators. 3. Models for generating contrastive claims using neural sequence models and constrained decoding. In the following sections, we describe the task methodology and data collection and processing. Next, we present neural models for contrastive claim generation and evaluate our work and present an error analysis. We then discuss related work in contrast/contradiction, argumentation, and generation before concluding. Previous work in claim generation (Bilu et al., 2015) focused on explicit negation to provide opposing claims. While negation plays an important role in argumentation (Apothloz et al., 1993), researchers found that explicit negation may result in incoherent responses (Bilu et al., 2015). Furthermore, recent empirical studies have shown that arguments that provide new content (Wachsmuth et al., 2018) tend to be more effective. While new concepts can be introduced in other ways by find-ing semantically relevant content, we may find it desirable to explicitly model contrast in order to control the output of the model as part of a rhetorical strategy, e.g. concessions (Musi, 2018). We thus develop a model that generates a contrastive claim given an input claim. Contrastive claims may differ in more than just viewpoint; they may also contain stylistic differences and paraphrases, among other aspects. We thus propose to model contrastive claims by controlling for context and maintaining the same text 1 Data and code available at github.com/chridey/fixedthat between pairs of contrastive claims except for the contrastive word or phrase. Much of the previous work in contrast and contradiction has examined the relationship between words or sentences. In order to understand when words and phrases are contrastive in argumentation, we need to examine them in context . For example, consider the claim Hillary Clinton should be president. A reasonable contrastive claim might be Bernie Sanders should be president. (rather than the explicit negation Hillary Clinton should not be president. ) In this context, Hillary Clinton and Bernie Sanders are contrastive entities as they were both running for president. However, for the claim Hillary Clinton was the most accomplished Secretary of State in recent memory. they would be unrelated. Consider also that we could generate the claim Hillary Clinton should be senator. This contrastive claim is not coherent given the context. Generating a contrastive claim then requires 1) identifying the correct substitution span and 2) generating a response with semantically relevant replacements. While some contrastive claims are not coherent, there are often multiple plausible responses, similar to tasks such as dialogue generation. For example, Donald Trump should be president is just as appropriate as Bernie Sanders should be president . We thus treat this as a dialogue generation task where the goal is to generate a plausible response given an input context. We obtain training data by scraping the social media site Reddit for comments containing the acronym FTFY . 2 FTFY is a common acronym meaning fixed that for you. 3 FTFY responses (hereafter FTFY) are used to respond to another comment by editing part of the parent comment (hereafter parent). Most commonly, FTFY is used for three categories of responses: 1) expressing a contrastive claim (e.g. the parent is Bernie Sanders for president and the FTFY is Hillary Clinton should be president ) which may be sarcastic (e.g. Ted Cruz for president becomes Zodiac 2 https://en.wiktionary.org/wiki/FTFY 3 https://reddit.zendesk.com/hc/en-us/articles/205173295-What-do-all-these-acronyms-mean killer for president ) 2) making a joke (e.g. This Python library really piques my interest vs. This really *py*ques my interest ), and 3) correcting a typo (e.g. This peaks my interest vs. piques ). In Section 3.2, we describe how we identify category 1 (contrastive claims) for modeling. To obtain historical Reddit data, we mined comments from the site pushshift.io for December 2008 through October 2017. This results in 2,200,258 pairs from Reddit, where a pair consists of a parent and an FTFY. We find that many of the top occurring subreddits are ones where we would expect strong opinions (/r/politics, /r/worldnews, and /r/gaming). To filter the data to only the type of response that we are interested in, we annotated comment pairs for contrastive claims and other types. We use our definition of contrastive claims based on contradiction, where both the parent and FTFY are a claim and they are unlikely to be beliefs held by the same speaker. A joke is a response that does not meaningfully contrast with the parent and commonly takes the form of a pun, rhyme, or oronym. A correction is a response to a typo, which may be a spelling or grammatical error. Any other pair is labeled as other, including pairs where the parent is not a claim. In order to identify contrastive claims, we selected a random subset of the Reddit data from prior to September 2017 and annotated 1993 comments. Annotators were native speakers of English and the Inter-Annotator Agreement using Kripen-dorff's alpha was 0.72. Contrast occurs in slightly more than half of the sampled cases (51.4%), with jokes (23.0%) and corrections (21.2%) comprising about one quarter each. We then train a binary classifier to predict contrastive claims, thus enabling better quality data for the generation task. To identify the sentence in the parent that the FTFY responds to and derive features for classification, we use an edit distance metric to obtain sentence and word alignments between the parent comment and response. As the words in the parent and response are mostly in the same order and most FTFYs contain significant overlap with the parent response, it is possible to find alignments by moving a sliding window over the parent. A sample of 100 comments verifies that this approach yields exact word alignments in 75 comments and exact sentence alignments in 93. Given these pairs of comments, we derive linguistic and structural features for training a binary classifier. For each pair of comments, we compute features for the words in the entire comment span and features from the aligned phrases span only (as identified by edit distance). From the aligned phrases we compute the character edit distance and character Jaccard similarity (both normalized by the number of characters) to attempt to capture jokes and typos (the similarity should be high if the FTFY is inventing an oronym or correcting a spelling error). From the entire comment , we use the percentage of characters copied as a low percentage may indicate a poor alignment and the percentage of non-ASCII characters as many of the jokes use emojis or upside-down text. In addition, we use features from GloVe (Pennington et al., 2014) word embeddings 4 for both the entire comment and aligned phrases . We include the percentage of words in the embedding vocabulary for both spans for both the parent and FTFY. The reason for this feature is to identify infrequent words which may be typos or jokes. We compute the cosine similarity of the average word embeddings between the parent and FTFY for both spans. Finally, we use average word embeddings for both spans for both parent and FTFY. As we want to model the generation of new content, not explicit negation, we removed any pairs where the difference was only stop words. The set of stop words includes all the default stop words in Spacy 5 combined with expletives and special tokens (we replaced all URLs and user-names). We trained a logistic regression classifier and evaluated using 4-fold cross-validation. We compare to a character overlap baseline where any examples with Jaccard similarity > 0 . 9 and edit distance < 0 . 15 were classified as non-contrastive. The goal of this baseline is to illustrate how much of the non-contrastive data involves simple or nonexistent substitutions. Results are shown in Table 1. Our model obtains an F-score of 80.25 for an 8 point absolute improvement over the baseline. After using the trained model to classify the remaining data, we have 1,083,797 Reddit pairs. We set aside 10,307 pairs from October 1-20, 2017 for 4 We found the 50-dimensional Wikipedia+Gigaword embeddings to be sufficient 5 spacy.io Model Precision Recall F-score Majority 51.4 100 67.5 Baseline 67.75 77.19 72.16 LR 74.22 87.60 80.25 Table 1: Results of Identifying Contastive Claims development and October 21-30 for test (6,773), with the remainder used for training. As we are primarily working with sentences, the mean parent length was 16.3 and FTFY length was 14.3. The resulting test FTFYs are naturally occurring and so do not suffer from annotation artifacts. At the same time, they are noisy and may not reflect the desired phenomenon. Thus, we also conducted an experiment on Amazon Mechanical Turk 6 (AMT) to obtain additional gold references, which are further required by metrics such as BLEU (Papineni et al., 2002). We selected 2,625 pairs from the 10 most frequent categories 7 (see Table 2). These categories form a three-level hierarchy for each subreddit and we use the second-level, e.g. for /r/pokemongo the categories are Pokemon, Video Games, and Gaming so we use Video Games. Before participating, each annotator was required to pass a qualification test five questions to gauge their knowledge of that topic. For the movies category, one question we asked was whether for the sentence Steven Spielberg is the greatest director of all time , we could instead use Stanley Kubrick or Paul McCartney . If they passed this test, the annotators were then given the parent comment and keywords (the subreddit and three category levels) to provide additional context. We obtained five new FTFYs for each parent and validated them manually to remove obvious spam or trivial negation (e.g. not or can't). Our goal of generating contrastive claims can be broken down into two primary tasks: 1) identifying the words in the original comment that should be removed or replaced and 2) generating the appropriate substitutions and any necessary context. Initially, we thus experimented with a modular approach by tagging each word in the parent and then using the model predictions to determine if we should copy, delete, or replace a segment with a new word or phrase. We tried the bi-directional LSTM-CNN-CRF model of Ma and Hovy (2016) and used our edit distance word alignments to obtain labels for copying, deleting, or replacing. However, we found this model performed slightly above random predictions, and with error propagation, the model is unlikely to produce fluent and accurate output. Instead, we use an end-to-end approach using techniques from machine translation. We use neural sequence-to-sequence encoder-decoder models (Sutskever et al., 2014) with attention for our experiments. The tokens from the parent are passed as input to a bi-directional GRU (Cho et al., 2014) to obtain a sequence of encoder hidden states h i . Our decoder is also a GRU, which at time t generates a hidden state s t from the previous hidden state s t  1 along with the input. When training, the input x t is computed from the previous word in the gold training data if we are in teacher forcing mode (Williams and Zipser, 1989) and otherwise is the prediction made by the model at the previous time step. When testing, we also use the model predictions. The input word w t may be augmented by additional features, as discussed in Section 4.2. In the baseline scenario x t = e ( w t ) where e is an embedding. The hidden state s t is then combined with a context vector h  t , which is a weighted combination of the encoder hidden states using an attention mechanism: h  t = (cid:88) i  it h i To calculate  ti , we use the attention of Luong et al. (2015) as this encourages the model to select features in the encoder hidden state which correlate with the decoder hidden state, which we want because our input and output are similar. Our experiments on the development data verified this, as Bahdanau attention (Bahdanau et al., 2015) performed worse. Attention is then calculated as:  it = exp( h Ti s t ) (cid:80) s (cid:48) exp( h Ts (cid:48) s t ) Finally, we make a prediction of a vocabulary word w by using features from the context and decoder hidden state with a projection matrix W and output vocabulary matrix V : P ( w ) = softmax ( V tanh( W [ s t ; h  t ] + b w ) + b v ) We explored using a copy mechanism (See et al., 2017) for word prediction but found it difficult to prevent the model from copying the entire input. Decoder Input: We evaluate two representations of the target input: as a sequence of words and as a sequence of edits . The sequence of words approach is the standard encoder-decoder setup. For the example parent Hillary Clinton for president 2020 and FTFY Bernie Sanders for president we would use the FTFY without modification. Schmaltz et al. (2017) found success modeling error correction using sequence-to-sequence models by representing the target input as a sequence of edits. We apply a similar approach to our problem, generating a target sequence by following the best path in the matrix created by the edit distance algorithm. The new target sequence is the original parent interleaved with DELETE-N to-kens that specify how many previous words to delete, followed by the newly generated content. For the same example, Hillary Clinton for president 2020 , the modified target sequence would be Hillary Clinton DELETE-2 Bernie Sanders for president 2020 DELETE-1 . Counter: Kikuchi et al. (2016) found that by using an embedding for a length variable they were able to control output length via a learned mechanism. In our work, we compute a counter variable which is initially set to the number of new content words the model should generate. During decoding, the counter is decremented if a word is generated that is not in the source input ( I ) or in the set of stop words ( S ) defined in Section 3.2. The model uses an embedding e ( c t ) for each count, which is parameterized by a count embedding matrix. The input to the decoder state in this scenario is x t = e ( w t , c t ) . At each time step, the count is computed by: c 0 = | O \ ( S  I ) | or desired count c t +1 = (cid:40) c t  1 , w t /  S  I and c t > 0 c t , otherwise where O is the set of gold output words in training. For the parent comment Hillary Clinton for president 2020 and FTFY Bernie Sanders for president , the decoder input is presented, with the time t in the first row of Table 3 and the inputs w t and c t in the second and third rows, respectively. At the start of decoding, the model expects to generate two new content words, which in this example it generates immediately and decrements the counter. When the counter reaches 0, it only generates stop or input words. Unlike the controlled-length scenario, at test time we do not know the number of new content words to generate. However, the count for most FTFYs is between 1 and 5, inclusive, so we can exhaustively search this range during decoding. We experimented with predicting the count but found it to be inaccurate so we leave this for future work. Subreddit Information: As the model often needs to disambiguate polysemous words, additional context can be useful. Consider the parent comment this is a strange bug. In a programming subreddit, a sarcastic FTFY might be this is a strange feature. However, in a Pokemon subreddit, an FTFY might be this is a strange dinosaur in an argument over whether Armaldo is a bug or a dinosaur. We thus include additional features to be passed to the encoder at each time step, in the form of an embedding g for each the three category levels obtained in Section 3.3. These embeddings are concatenated to the input word w t at each timestep, i.e. x t = e ( w t , g 1 t , g 2 t , g 3 t ) . We use a negative log likelihood objective function LNLL =  log (cid:80) t  1: TP ( w  t ) , where w  t is the gold token at time t , normalized by each batch. We also include an additional loss term that uses the encoder hidden states to make a binary prediction over the input for whether a token will be copied or inserted/deleted. For the example from Section 4.2, the target for Hillary Clinton for president 2020 would be 0 0 1 1 0 . This encourages the model to select features that indicate whether the encoder input will be copied to the output. We use a 2-layer multi-layer perceptron and a binary cross-entropy loss LBCE . The joint loss is then: L = LNLL +  LBCE 4.4 Decoding We use beam search for generation, as this method has proven effective for many neural language generation tasks. For the settings of the model that require a counter, we expand the beam by count m so that for a beam size k we calculate k  m states. Filtering : We optionally include a constrained decoding mode where we filter the output based on the counter; when c t > 1 the end-of-sentence (EOS) score is set to  and when c t = 0 the score of any word w  V \ ( S  I ) is set to  . The counter c t is decremented at every time step as in Section 4.2. In other words, when the counter is zero, we only allow the model to copy or generate stop words. When the counter is positive, we prevent the model from ending the sentence before it generates new content words and decrements the counter. The constrained decoding is possible with any combination of settings, with or without the counter embedding. We used Pytorch (Paszke et al., 2017) for all experiments. We used 300-dimensional vectors for the word embedding and GRU layers. The count embedding dimension was set to 5 with m = 5 and k = 10 for decoding. The category embedding dimensions were set to 5, 10, and 25 for each of the non-subreddit categories. We also set  = 1 for multi-task learning. We used the Adam optimizer (Kingma and Ba, 2015) with settings of  1 = 0 . 9 ,  2 = 0 . 999 , (cid:15) = 10  8 and a learning rate of 10  3 decaying by  = 0 . 1 every epoch. We used dropout (Srivastava et al., 2014) on the embeddings with a probability of 0.2 and teacher forcing with 0.5. We used a batch size of 100 with 10 epochs, selecting the best model on the development set based on perplexity. We set the minimum frequency of a word in the vocabulary to 4. For training, development, and testing we use the data described in Section 3.3. The test reference data consists of the Reddit FTFYs and the FTFYs generated from AMT. We evaluate our models using automated metrics and human judgments. Automated metrics should reflect our joint goals of 1) copying necessary context and 2) making appropriate substitutions. To address point 1, we use BLEU-4 as a measure of similarity between the gold FTFY and the model output. As the FTFY may contain significant overlap with the parent, BLEU indicates how well the model copies the appropriate context. As BLEU reflects mostly span selection rather than the insertion of new content, we need alternative metrics to address point 2. However, addressing point 2 is more difficult due to the variety of possible substitutions, including named entities. For example, if the parent comment is jaguars for the win! and the gold FTFY is chiefs for the win! but the model produces cowboys for the win! (or any of 29 other NFL teams), most metrics would judge this response incorrectly even though it would be an acceptable response. Thus we present results using both automated metrics and human evaluation. As an approximation to address point 2, we attempt to measure when the model is making changes rather than just copying the input. To this end, we present two additional metrics novelty , a measure of whether novel content (non-stop word) tokens are generated relative to the parent comment, and partial match , a measure of whether the novel tokens in the gold FTFY match any of the novel tokens in the generated FTFY. To provide a reference point, we find that the partial match between two different gold FTFYs (Reddit and AMT) was 11.4% and BLEU was 47.28, which shows the difficulty of automatic evaluation. The scores are lower than expected because the Reddit FTFYs are noisy due to the process in Section 3.2. This also justifies obtaining the AMT FTFYs. Results are presented in Table 4. The baseline is a sequence-to-sequence model with attention. For other components, the counter embedding is referred to as COUNT, the category/subreddit embeddings as SUB, the sequence of edits as EDIT, and the multi-task copy loss as COPY. The models in the top half of the table use constrained decoding and those in the bottom half are unconstrained, to show the learning capabil-Reddit AMT Model Novelty BLEU-4 % Match BLEU-4 % Match C on s t r a i n e d Baseline 79.88 18.81 4.67 40.14 10.06 COUNT 89.69 22.61 4.72 47.55 12.55 COUNT + SUB + COPY 90.45 23.13 4.83 50.05 14.92 EDIT 64.64 16.12 3.37 35.48 7.33 EDIT + COUNT + SUB + COPY 82.96 19.37 4.23 42.69 11.62 U n c on s t r a i n e d Baseline 3.34 7.31 0.73 25.83 0.68 COUNT 16.19 8.51 1.95 27.68 2.36 COUNT + SUB + COPY 16.26 9.62 1.93 31.23 3.81 EDIT 7.97 35.41 1.57 74.24 1.56 EDIT + COUNT + SUB + COPY 39.99 32.59 3.25 67.56 6.25 Table 4: Automatic Evaluation ities of the models. For each model we compute statistical significance with bootstrap resampling (Koehn, 2004) for the constrained or unconstrained baseline as appropriate and we find the COUNT and EDIT models to be significantly better for constrained and unconstrained decoding, respectively ( p < 0 . 005 ). Under constrained decoding, we see that the COUNT + SUB + COPY model performs the best in all metrics, although most of the performance can be attributed to the count embedding. When we allow the model to determine its own output, we find that EDIT + COUNT performs the best. In particular, this model does well at understanding which part of the context to select, and even does better than other unconstrained models at selecting appropriate substitutions. However, when we combine this model with constrained decoding, the improvement is smaller than for the other settings. We suspect that because the EDIT model often needs to generate a DELETE-N token before a new response, these longer-term dependencies are hard to capture with constrained decoding but easier if included in training. We also conducted a human evaluation of the model output on the same subset of 2,625 examples described in Section 3.3. We performed an additional experiment on AMT where we asked annotators to rate responses on fluency, coherence, and contrast. Fluency is a measure of the quality of the grammar and syntax and the likelihood that a native English speaker would utter that statement. Coherence is a measure of whether the response makes sense, is semantically meaningful, and would be usable as a response to a claim. Contrast is a measure of how much the response contradicts the original comment. We specified that if the response is different but does not provide a contrasting view it should receive a low rating. Previous work (Bilu et al., 2015) used fluency, clarity/usability (which we combine into co-herence), and opposition (where we use contrast). We used a Likert scale where 5 is strongly agree and 1 is strongly disagree. We used the same data and qualification test from Section 3.3 for each category and used three annotators per example. We asked annotators to judge 4 different pairs: 3 model outputs and the gold Reddit 8 FTFYs for comparison. We include the baseline, the baseline with constrained decoding, and the best constrained model (COUNT + SUB + COPY) according to BLEU and partial match. We verified that the annotators understood how to rate contrast by examining the distribution of responses: the annotators selected option 3 (neither) 15% of the time and preferred to select either extreme, 5 (21%) or 1 (27%). Results are presented in Table 5, showing a clear preference for the best model. Note the degradation in fluency for the constrained baseline, as the model is prevented from generating the EOS token and may repeat tokens up to the 8 We did not evaluate the AMT FTFYs as these were generated by the same pool of annotators. Parent: ah yes the wonders of the free market Model: ah yes the wonders of government intervention Parent: i know that this is an unofficial mod , but xp is the best os for this machine Model: linux is the best os for this machine Parent: that 's why it 's important to get all your propaganda from infowars and brietbart Model: propaganda from fox news outlets Table 6: Model Output maximum length. We provide three examples of the model output in Table 6 with the first and third from the News and Politics category, demonstrating how the model handles different types of input. In the first example, the contrast is between allowing markets to regulate themselves versus an increased role of government. In the second example, the contradiction is due to the choice of operating system. In the third (invalid) example, the model responds to a sarcastic claim with another right-wing news organization; this response is not a contradiction since it is plausible the original speaker would also utter this statement. We conduct an error analysis by selecting 100 responses where the model did not partially match any of the 6 gold responses and we found 6 main types of errors. One error is the model identifying an incorrect substitution span while the human responses all selected a different span to replace. We noticed that this occurred 5 times and may require world knowledge to understand which tokens to select. For example, in response to the claim Hillary Clinton could have been president if not for robots , the model generates Donald Trump in place of Hillary Clinton , whereas the gold responses generate humans / votes / Trump's tweets in place of robots . Another type of error is when the responses are not coherent with the parent and the language model instead determines the token selection based on the most recent context (11 cases). For example, given the claim bb-8 gets a girlfriend and poe still does n't have a girlf :') the Reddit FTFY has boyf instead of girlf whereas the model generates ... and poe still does n't have a clue what i 'm talking about . We also found examples where the model chose poorly due to un-filtered jokes or errors in the training data (12 in total). In 15 cases, due to the constrained decoding the model repeated a word until the maximum length or appended an incoherent phrase. For the most common error, the model made a substitution that was not contrasting as in Table 6 (19 examples). Finally, we found 38 of the samples were valid responses, but did not match the gold, indicating the difficulty of automatic evaluation. For example, in response to the claim Nintendo is the only company that puts customers over profits , the model replaces Nintendo with Rockstar (both video game companies) while the gold FTFYs had other video game companies. Understanding contrast and contradiction is key to argumentation as it requires an understanding of differing points-of-view. Recent work examined the negation of claims via explicit negation (Bilu et al., 2015). Other work investigated the detection of different points-of-view in opinionated text (Al Khatib et al., 2012; Paul et al., 2010). Wachsmuth et al. (2017; 2018) retrieved arguments for and against a particular stance using online debate forums. In non-argumentative text, researchers predicted contradictions for types such as negation, antonyms, phrasal, or structural (de Marneffe et al., 2008) or those that can be expressed with functional relations (Ritter et al., 2008). Other researchers have incorporated entailment models (Kloetzer et al., 2013) or crowd-sourcing methods (Takabatake et al., 2015). Contradiction has also become a part of the natural language inference (NLI) paradigm, with datasets labeling contradiction, entailment, or neutral (Bow-man et al., 2015a; Williams et al., 2018). The increase in resources with contrast and contradiction has resulted in new representations with contrastive meaning (Chen et al., 2015; Nguyen et al., 2016; Vulic, 2018; Conneau et al., 2017). Most of this work has focused on identifying contrast or contradiction while we aim to generate contrast. Furthermore, while contradiction and contrast are present in these corpora, we obtain distant-labeled alignments for contrast at the word and phrase level. Our dataset also includes contrastive concepts and entities while other corpora primarily contain antonyms and explicit negation. Contrast also appears in the study of stance, where the opinion towards a target may vary. The SemEval 2016 Stance Detection for Twitter task (Mohammad et al., 2016) involved predicting if a tweet favors a target entity. The Interpretable Semantic Similarity task (Agirre et al., 2016) called to identify semantic relation types (including opposition) between headlines or captions. Target-specific stance prediction in debates is addressed by Hasan and Ng (2014) and Walker et al. (2012). Fact checking can be viewed as stance toward an event, resulting in research on social media (Lend-vai and Reichel, 2016; Mihaylova et al., 2018), politician statements (Vlachos and Riedel, 2014), news articles (Pomerleau and Rao, 2017), and Wikipedia (Thorne et al., 2018). In computational argumentation mining, identifying claims and other argumentative components is a well-studied task (Stab and Gurevych, 2014). Daxenberger et al. (2017) and Schulz et al. (2018) developed approaches to detect claims across across diverse claim detection datasets. Recently, a shared task was developed for argument reasoning comprehension (Habernal et al., 2018). The best system (Choi and Lee, 2018) used models pre-trained on NLI data (Bowman et al., 2015b), which contains contradictions. While this work is concerned with identification of argumentative components, we propose to generate new claims. In the field of argument generation, Wang and Ling (2016) train neural abstractive summarizers for opinions and arguments. Additional work involved generating opinions given a product rating (Wang and Zhang, 2017). Bilu and Slonim (2016) combine topics and predicates via a template-based classifier. This work involves the generation of claims but in relation to a topic. Other researchers generated political counter-arguments supported by external evidence (Hua and Wang, 2018) and generating argumentative dialogue by maximizing mutual information (Le et al., 2018). This research considers end-to-end argument generation, which may not be coherent, whereas we focus specifically on contrastive claims. We presented a new source of over 1 million contrastive claim pairs that can be mined from social media sites such as Reddit. We provided an analysis and models to filter noisy training data from 49% down to 25%. We created neural models for generating contrastive claims and obtained significant improvement in automated metrics and human evaluations for Reddit and AMT test data. Our goal is to incorporate this model into an argumentative dialogue system. In addition to generating claims with a contrasting view, we can also retrieve supporting evidence for the newly-generated claims. Additionally, we plan to experiment with using our model to improve claim detection (Daxenberger et al., 2017) and stance prediction (Bar-Haim et al., 2017). Our model could be used to generate artificial data to enhance classification performance on these tasks. To improve our model, we plan to experiment with retrieval-based approaches to handle low-frequency terms and named entities, as sequence-to-sequence models are likely to have trouble in this environment. One possibility is to incorporate external knowledge with entity linking over Wikipedia articles to find semantically-relevant substitutions. Another way to improve the model is by introducing controllable generation. One aspect of controllability is intention; our model produces contrastive claims without understanding the view of the original claim. Category embeddings partially address this issue (some labels are Liberal or Conservative), but labels are not available for all views. Going forward, we hope to classify the viewpoint of the original claim and then generate a claim with a desired orientation. Furthermore, we hope to improve on the generation task by identifying the types of claims we encounter. For example, we may want to change the target of the claims in some claims but in others change the polarity. We also plan to improve the dataset by improving our models for contrastive pair prediction to reduce noise. Finally, we hope that this dataset proves useful for related tasks such as textual entailment (providing examples of contradiction) and argument comprehension (providing counterexamples of arguments) or even unrelated tasks like humor or error correction. We would like to thank the AMT annotators for their work and the anonymous reviewers for their valuable feedback. The authors also thank Alyssa Hwang for providing annotations and Smaranda Muresan, Chris Kedzie, Jessica Ouyang, and Elsbeth Turcan for their helpful comments.
abstain abstain abstain method objective objective abstain abstain abstain objective abstain method abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain method method abstain objective method abstain objective objective objective method method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other method abstain abstain abstain method method abstain abstain abstain method abstain abstain method abstain abstain abstain method method method method method method abstain method method abstain method method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method method method method method method abstain method method method abstain abstain method method method method abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain method abstain method method abstain abstain method method method method abstain method method abstain method method abstain abstain method method abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain other result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other other objective objective abstain other other other other other other other other other objective other other other abstain other method objective method result objective objective result method result abstain abstain method abstain method result abstain result method other other