We propose an unsupervised approach for sarcasm generation based on a non-sarcastic input sentence. Our method employs a retrieve-and-edit framework to instantiate two major characteristics of sarcasm: reversal of valence and semantic incongruity with the context, which could include shared commonsense or world knowledge between the speaker and the listener. While prior works on sarcasm generation predominantly focus on context incongruity, we show that combining valence reversal and semantic incongruity based on commonsense knowledge generates sarcastic messages of higher quality based on several criteria. Human evaluation shows that our system generates sarcasm better than human judges 34% of the time, and better than a reinforced hybrid baseline 90% of the time. Studies have shown that the use of sarcasm or verbal irony, can increase creativity on both the speakers and the addressees (Huang et al., 2015), and can serve different communicative purposes such as evoking humor and diminishing or enhancing critique (Burgers et al., 2012). Thus, developing computational models that generate sarcastic messages could impact many downstream applications, such as better conversational agents and creative or humorous content creation. While most computational work has focused on sarcasm detection (Davidov et al., 2010; Gonzalez-Ibanez et al., 2011; Riloff et al., 2013; Ghosh et al., 2015; Joshi et al., 2015b; Muresan et al., 2016; Ghosh and Veale, 2017; Ghosh et al., 2017, 2018), research on sarcasm generation is in its infancy (Joshi et al., 2015a; Mishra et al., 2019). Sarcasm generation  The research was conducted when the author was at USC/ISI. is a challenging problem since the generated utterance should have at least five characteristics (a.k.a. sarcasm factors) (Burgers et al., 2012): 1) be evaluative; 2) be based on a reversal of valence between the literal and intended meaning; 3) be based on a semantic incongruity with the context, which can include shared commonsense or world knowledge between the speaker and the addressee; 4) be aimed at some target, and 5) be relevant to the communicative situation in some way. To simplify the problem, we focus on the task of generating a sarcastic utterance starting from a non-sarcastic utterance that conveys the speaker's intended meaning and that is evaluative. Consider the examples in Table 1. Given the literal input I hate getting sick from fast food or I inherited unfavorable genes from my mother, our task is to generate a sarcastic message that would convey this intended literal meaning. In this simplifying task, we are not concerned with the fifth characteristic, while the first and to some degree, the fourth are specified by the input (literal) utterances. Given the lack of training data for the sarcasm generation task, we propose a novel unsupervised approach that has three main modules guided by the above mentioned sarcasm factors: 1. Reversal of Valence: To generate sarcastic utterances that satisfy the second characteristic we identify the evaluative word and use negation or lexical antonyms to generate the sarcastic utterance by reversing the valence (Section 4.1). For example, given, I hate getting sick from fast food this module will generate I love getting sick from fast food (GenSarc1 in Table 1). 2. Retrieval of Commonsense Context: Adding commonsense context could be important to make explicit the semantic incongruity factor (e.g., GenSarc4 vs. GenSarc3 in Table 1), or could enhance the humorous effect of the generated sarcastic message (e.g., GenSarc2 vs. GenSarc1 in Table 1). We propose an approach where retrieved relevant commonsense context sentences are to be added to the generated sarcastic message. At first, we use a pre-trained language model fine-tuned on the ConceptNet (Speer et al., 2017) called COMET (Bosselut et al., 2019) to generate relevant commonsense knowledge. COMET gives us that, inherited unfavorable genes from my mother causes to be ugly or that getting sick from fast food causes stomach ache (Section 4.2.1). The derived commonsense concept is then used to retrieve relevant sentences  from a corpus  that could be added to the sentence obtained through reversal of valence (e.g., Stomach ache is just an additional side effect in Table 1) (Section 4.2.2). 3. Ranking of Semantic Incongruity: The previous module generates a list of candidate commonsense contexts. Next, we measure contradiction between each of these commonsense contexts and the sentence generated by the reversal of valence approach (module 1) and select the commonsense context that received the highest contradiction score. Finally, we concatenate the selected context to the sentence obtained through reversal of valence. Here, conceptually, contradiction detection is aimed to capture the semantic incongruity between the output of valence reversal and its context. We test our approach on 150 non-sarcastic utterances randomly sampled from two existing data sets. We conduct human evaluation using several criteria: 1) how sarcastic is the generated message; 2) how humorous it is; 3) how creative it is; and 4) how grammatical it is. Evaluation via Amazon's Mechanical Turk (MTurk) shows that our system is better 34% of the time compared to humans and 90% of the time compared to a recently published reinforced hybrid baseline (Mishra et al., 2019). We also present a thorough ablation study of several variations of our system demonstrating that incorporating more sarcasm factors (e.g., reversal of valence, commonsense context, and semantic incongruity) lead to higher quality sarcastic utterances. We make the code and data from our experiments publicly available. 1 2 Related Work 2.1 Sarcasm Generation Research on sarcasm generation is in its infancy. Joshi et al. (2015a) proposed SarcasmBot , a sarcasm generation system that implements eight rule-based sarcasm generators, each of which generates a certain type of sarcastic expression. Peled and Reichart (2017) introduced a novel task of sarcasm interpretation, defined as the generation of a non-sarcastic utterance conveying the same message as the original sarcastic one. They use supervised machine translation models for the same in presence of parallel data. However, it is impractical to assume the existence of large corpora for training supervised generative models using deep neural nets; we hence resort to unsupervised approaches. Mishra et al. (2019) employed reinforced neural seq2seq learning and information retrieval based approaches to generate sarcasm. Their models are trained using only unlabeled non-sarcastic and sarcastic opinions. They generated sarcasm as a disparity between positive sentiment context and negative situational context. We, in contrast, model sarcasm using semantic incongruity with the context which could include shared commonsense or world knowledge. Prior works looked into unsupervised text style/sentiment transfer (Shen et al., 2017; Fu et al., 2017; Li et al., 2018), which transfers a sentence from one style to another without changing the content. This is relevant to the reversal of valence for sarcasm generation. However, these transformations are mainly at the lexical and syntax levels rather than pragmatic level; in contrast, sarcastic utterances often include additional information associated with the context they occur (Regel, 2009), which is beyond text style/sentiment transfer. The study of irony and sarcasm are closely related as sarcasm is defined as, the use of verbal irony to mock someone or show contempt. Van Hee et al. (2018) addressed the challenge of modeling implicit or prototypical sentiment in the framework of automatic irony detection. They first manually annotated stereotypical ironic situations (e.g., flight delays) and later addressed the implicit sentiment held towards such situations automatically by using both a lexico-semantic commonsense knowledge base and a data-driven method. They however used it for irony detection, while we are focused on sarcasm generation. 2 3 Sarcasm Factors Used in Generation A sarcastic utterance must satisfy the sarcasm factors, i.e., the inherent characteristics of sarcasm (Attardo, 2000; Burgers et al., 2012). In this research, we leverage the use of two particular factors to generate sarcasm. One is the reversal of valence and the other is the semantic incongruity with the context , which could include shared commonsense or world knowledge between the speaker and the hearer. The first key sarcasm factor is the reversal of valence between the literal and the intended meaning (Burgers et al., 2012). Reversal of valence can be achieved in two ways: when the literal meaning of the sarcastic message is positive (e.g., that is a great outfit if the outfit is ugly) or when the literal 2 While we do not directly model the negative intent in sarcasm, the generated output could lead to sarcastic messages rather than just ironic depending on the initial target given in the non-sarcastic message (E.g a sample generation Our politicians have everything under control. The nation is in danger of falling into anarchy.) meaning is negative (e.g., that is an ugly dress if the dress is really beautiful). Arguably, the former is more likely to appear in sarcastic utterances. As the intended meaning is generally the opposite of its literal meaning in sarcastic utterances (Gibbs, 1986), using lexical antonym of negative sentiment words or negation can be used to convert a non-sarcastic utterance to its sarcastic version. For example, given a non-sarcastic utterance Zero visibility in fog makes driving difficult , one could identify the evaluative negative word difficult and replace it with its antonym easy , thereby converting the utterance to the sarcastic Zero visibility in fog makes driving easy . Likewise, Drunk driving should be taken seriously can be converted to its sarcastic counterpart, Drunk driving should not be taken seriously by using negation. We propose a generation approach that is able to capture the reversal of valence (Section 4.1). The second sarcasm factor, semantic incongruity, appears between the literal evaluation and the context, as in the example I love getting sick from fast food, where we have semantic incongruity between the positive word love and the negative situation getting sick. However, often, the negative situation is absent from the utterance, and thus additional pragmatic inference is needed to understand the sarcastic intent. For example, the listener might miss the sarcastic intent in zero visibility in fog makes driving easy, where the speaker meant to convey that it can cause accidents . Adding suffered three cracked ribs in an accident. makes the sarcastic intent more explicit, while maintaining the acerbic wit of the speaker. In the next section, we propose a novel generation approach that incorporates such relevant commonsense knowledge as context for semantic incongruity (Section 4.2 and Section 4.3). An overview of the sarcasm generation pipeline is shown in Figure 1. In this section, we detail the three main modules that are designed to instantiate the key sarcasm factors. As sarcasm is a type of verbal irony used to mock or convey contempt, in most sarcastic messages we encounter a positive sentiment towards a nega-Figure nega-Figure 1: Our complete pipeline for sarcasm generation. The components with highlighted background denote Reversal of Valence, Retrieval of Commonsense Context and Ranking based on Semantic Incongruity respectively tive situation (i.e., ironic criticism (Kreuz and Link, 2002)). This observation is also supported by research on sarcasm detection, particularly on social media. Hence, for our sarcasm generation task, we focus on transforming a literal utterance with negative valence into positive valence. To implement the reversal of valence, as highlighted in the yellow background in Figure 1, we first identify the evaluative words and replace them with their lexical antonyms using WordNet (Miller, 1995). As we expect the evaluative words to be negative words, we rely on the word level negative scores obtained from SentiWordNet (Esuli and Sebastiani, 2006). In the absence of words with negative polarity, we check if there is the negation word not or words ending with n't and remove these words. In case there are both negative words and not (or words ending in n't ), we handle only one of them. Given the non sarcastic example zero visibility in fog makes driving difficult  shown in Figure 1 and which we use as our running example, the reversal of valence module generates zero visibility in fog makes driving easy  . As discussed before, a straightforward reversal of valence might not generate sarcastic messages that display a clear semantic incongruity, and thus, additional context is needed. We propose an approach to retrieve relevant context for the sarcastic message based on commonsense knowledge. First, we generate commonsense knowledge based on Con-cepNet (e.g., driving in zero visibility causes accidents) (Section 4.2.1). Second, we retrieve candidate context sentences that contain the commonsense concept from a retrieval corpus (Section 4.2.2) and edit them for grammatical consistency with the input message (Section 4.2.3). We extract nouns, adjectives, adverbs, and verbs from the non-sarcastic input messages and feed them as input to COMET (Bosselut et al., 2019) model to generate commonsense knowledge (high-lighted in green background in Figure 1). COMET is an adaptation framework for constructing commonsense knowledge based on pre-trained language models. It initiates with a pre-trained GPT (Radford et al., 2018) model and fine-tune on commonsense knowledge tuples (in our case, ConceptNet (Speer et al., 2017)). These tuples provide COMET with the knowledge base structure and relations that must be learned, and COMET adapts the representations that the language model learned from the pre-training stage to add novel nodes to the seed knowledge graph. Our work only leverages the causes relation. For instance, from our running example, we first remove the stopwords and then extract nouns, adjectives, adverbs, and verbs including the terms zero , visibility , fog , makes driving , and difficult to feed to COMET as inputs. In turn, COMET returns the probable causes with their probability scores. For the running example, COMET returns with the highest probability that these terms may cause an accident (illustrated in Figure 2). Commonsense Concepts Once we obtain the most probable output from COMET, the next step is to retrieve sentences containing the commonsense word or phrase from a retrieval corpus. We impose several constraints: (a) the retrieved sentences should contain the commonsense concept at the beginning or at the end; (b) sentence length should be less than twice the number of tokens in the non-sarcastic input to keep a consistency between the length of the non-sarcastic input and its sarcastic version. If none of the commonsense phrase is present in the retrieval corpus, we retrieve sentences containing the nouns within the top most phrase. For example, if COMET yields microwave burger awful causes the phrase food to spoil , and this phrase does not appear in any sentence in the retrieval corpus, we search for food and later replace it in the retrieved sentence with food to spoil . COMET often returns output with common phrases such as you to be , you to get , person will be , you have which we also removed while keeping the main content word (i.e the commonsense concept) We use Sentencedict.com, an online sentence dictionary as the retrieval corpus, where one can find high quality sentences for almost every word obeying the above constraints. 3 4.2.3 Grammatical Consistency We first check whether the retrieved sentences are consistent with the non-sarcastic input in terms of the pronouns. If the pronouns are mismatched, then we modify the pronoun of the retrieved sentence to match the pronoun of the non-sarcastic input. In case, the non-sarcastic input does not have any pronoun, but the retrieved sentence does, we simply change that pronoun to I. For example, if the non-sarcastic input sentence is Ignoring texts is literally the worst part of communication. and the retrieved commonsense sentence is  He has never suffered the torment of rejection. , we modify the retrieved sentence to  I have never suffered the torment of rejection. to have consistency among the pronoun use. After correcting the pronouns and proper names (in the same way as pronoun correction), we feed the corrected sentences into the Neural Grammatical Error Corrections System 3 https://sentencedict.com/ (Zhao et al., 2019) to correct any pronoun or gender specific errors introduced by the replacements. After the grammatical error correction, the next step is to select the best context sentence from the retrieved results. Since we expect the context sentences to be incongruous with the sentence generated by the reversal of valence approach (Section 4.1), we rank the context sentences by semantic incongruity scores and select the best candidate. We frame the problem of semantic incongruity based on the Natural Language Inference (NLI) (Bowman et al., 2015) task. The Multi-Genre NLI (Williams et al., 2018) covers a range of genres of spoken and written text, and supports a distinctive cross-genre generalization, making it an ideal choice as our NLI Dataset. We first fine-tune RoBERTa-large (Liu et al., 2019), a state-of-the-art pre-trained language model for a 3-way classifica-tion (i.e., contradiction, entailment, and neutral) by training on the Multi-NLI dataset. Next, for each retrieved sentence, we treat it as the premise and the sentence generated by the reversal of valence as the hypothesis , and thus, obtain a contradiction score from the trained model. Finally, the scores obtained for the contradiction class are used as a proxy for the degree of semantic incongruity and we select the context with the highest score. Figure 1 shows the region with light purple background as our incongruity ranking module. We use the pre-trained COMET model 4 for commonsense reasoning with a greedy decoding of five to generate a commonsense phrase and return the topmost that has no lexical overlap with the input. If the generated phrase contains stopwords in the beginning we remove them. For incorporating semantic incongruity, we use the RoBERTa-large model with 355M parameters and fine-tune on MNLI. For grammatical error correction model, we use an open source pre-trained model. 5 5 Experimental Setup 5.1 Dataset Ghosh et al. (2020) released a dataset of 4,762 pairs of speakers sarcastic messages and hearers interpretations by conducting a crowdsourcing experiment. Both datasets were collected using the hashtag #sarcasm from Twitter. We merge these two datasets and choose non-sarcastic utterances no longer than 15 words. For each literal non-sarcastic utterance we also keep the corresponding gold sarcastic message, which is useful for evaluation and comparison purposes. We randomly select 150 utterances as part of the test set (i.e., five times more than the size of the test data in Mishra et al. (2019)), while assuring such utterances do not contain high lexical overlap. We allow this constraint to evaluate how our method(s) deal with diverse data. 1. Full Model (FM) : This model consists of all the three modules aimed at capturing reversal of valence, commonsense context, and semantic incongruity, respectively. 2. Reversal of Valence (RV) : This model relies only on the reversal of valence component. 3. No Reversal of Valence (NoRV) : This model only retrieves commonsense context and ranks them based on semantic incongruity. 4. No Semantic Incongruity (NSI) : This model relies only on the reversal of valence and retrieval of commonsense context, without ranking based on semantic incongruity. A randomly selected retrieved sentence is used. 5. MTS2019 : We make use of the model released by Mishra et al. (2019) as it is the state-of-the-art sarcasm generation system. 6 6. Human (Gold) Sarcasm : As described in Section 5.1, we have gold sarcasm created by humans for every non-sarcastic utterance. BLEU (Papineni et al., 2002) is one of the most widely used automatic evaluation metric for generation tasks such as Machine Translation. However, for creative text generation, it is not ideal to expect significant n-gram overlaps between the machine-generated and the gold-standard utterances. Hence, we performed a human evaluation. We evaluate a total of 900 generated utterances since our ablation study consisted of six different systems with 150 utterances each. Sarcasm is often linked with intelligence, creativity, and wit; thus we propose a set of 4 criteria to evaluate the generated output: (1) Creativity (How creative are the utterances ?), (2) Sarcasticness (How sarcastic are the utterances ?), (3) Humour (How funny are the sentences ?) (Skalicky and Crossley, 2018), and (4) Grammaticality (How grammatical are the sentences ?). We design a MTurk task where Turkers were asked to rate outputs from all the six systems. Each Turker was given the non-sarcastic utterance as well as a group of sarcastic utterances generated by all the six systems (randomly shuffled). Each criteria was rated on a scale from 1 ( not at all ) to 5 ( very ). Finally, each utterance was rated by three individual Turkers. 55, 59, 66, and 60 Turkers 6 https://github.com/TarunTater/sarcasm generation System Sarcasticness Creativity Humor Grammaticality State-of-the-art (Mishra et al., 2019) 1.63 1.60 1.50 1.46 Human Generated 3.57 3.16 3.18 3.98 Reversal of Valence (RV) 3.00 2.80 2.72 4.29 No Reversal of Valence (NoRV) 1.79 2.28 2.09 3.91 No Semantic Incongruity (NSI) 3.04 2.99 2.90 3.68 Full Model (FM) 3.23* 3.24 3.08* 3.69 Table 2: Average scores for generated sarcasm from all systems as judged by the Turkers. attempted the HITs (inter-annotator agreement of 0.59, 0.53, 0.47 and 0.66 for the tasks on creativity, sarcasticness, humour and grammaticality, respectively using Spearman's correlation coefficient). Table 2 presents the scores for the above mentioned metrics of different systems averaged over 150 test utterances. Our full model as well as the variations that ablated some components improve over the state-of-the-art (Mishra et al., 2019) on all the criteria. The ablation in Table 2 shows that our full model is superior to individual modules in terms of sarcasticness, creativity and humor. For grammaticality, we observe that the Turkers scored shorter sentences higher (e.g., RV), which also explains why NoRV model received a higher score than the full model. NoRV otherwise performed worse than all the other variations. In terms of creativity, our full model attains the highest average scores over all the other models including sarcastic utterances composed by humans. For grammaticality, the reversal of valence model is the best, even better than human gener-Figure 3: Pie chart comparing the success rate of all the variations of our model. ated ones. The performance of the full model is the second best in terms of the sarcasticness and humor, only slightly worse than human-generated sarcasm, showing the effectiveness of our approach that captures various factors of sarcasm. Table 3 displays the pairwise comparisons between the full model (FM) and human generated sarcasm, and FM and Mishra et al. (2019), respectively. Given a pair of inputs, we decide win/lose/tie by comparing the average scores (over three Turkers) of both outputs. We see that FM dominates Mishra et al. (2019) on all the metrics and human-generated sarcasm on the creativity metric. For sarcasticness, although humans are better, the FM model still has a 34% winning rate. We focus our ablation study on the metric of sarcasticness, as we consider this as the main criterion for the success of generating sarcasm. As shown in Figure 3, our best model (FM) outperforms individ-Non Sarcastic System Sarcasm S C H G I inheritedunfavorable genes from my mother. ual ablation modules. We filtered out 60 examples from the 150 with no ties. The ablation component employing just Reversal of Valence is second best for sarcasticness according to Figure 3. Further, to understand the extent to which ranking the retrieved sentence based on the degree of incongruity helped generate better sarcasm, we took the outputs from FM and NSI for comparisons. Out of the 150 utterances, 119 times there was no tie. Our best model (FM) wins 66% of the time while the NSI model wins 34% of the cases. Table 4 demonstrates several generation outputs from different modules associated with human ratings for different criteria. We notice that often one of our modules generate better sarcasm than humans. For instance, for the first and the second example in Table 4, all of FM, RV and NSI are better than human generated sarcasm. In general, the generations from the FM model are more humorous, which is also an useful criterion to evaluate sarcasm besides sarcasticness (Skalicky and Crossley, 2018). We also observe that Turkers consistently rated generations from the FM model more sarcastic than the NSI model suggesting that there is a correlation between human scores of sarcasticness and incongruity. To support this observation, we took the contradiction scores from the RoBERTa model for both best ranked retrieved sentences (FM) and the randomly selected retrieved sentences (NSI). We then computed a correlation between the sarcasticness scores given by the humans and the automatic contradiction scores for both the best ranked retrieved sentences (FM) and the randomly selected retrieved sentences (NSI). For FM model we obtain a higher Pearson correlation coefficient compared to NSI suggesting the important role of incongru-ency for sarcasm. While our best model combining different sarcasm factors does outperform the system with individual factors, there are sometimes exceptions. We notice, in few cases, the simple reversal of valence (RV) strategy is enough to generate sarcasm. For instance, for the literal input It is not fun to date a drug addict, just removing the negation word leads to a full score on sarcasticness without the additional commonsense module. Future work would include building a model that can decide whether just the RV strategy is sufficient or if we need to add additional commonsense context to it. Although incorporating incongruity ranking is useful, there are several cases when a randomly retrieved message may obtain better sarcasticness score. Table 5 presents such an example. Even though the retrieved message Please stop whirling me round; it makes me feel sick. scores lower than The very thought of it makes me feel sick., in terms of incongruity with respect to I love being put in the hospital for dehydration, the former received a higher sarcasticness score that suggests the incongruity scores obtained from NLI are not perfect. The ordering of the commonsense context and the valence reversed sentence is predetermined in our generation. Specifically, we always append the retrieved commonsense context after the valence reversed output. Changing the order can sometimes make the sarcasm better and more humorous. The reason for our current ordering choice is that we always treat the valence reversed version as hypothesis and the commonsense retrieved sentence as premise for the NLI model. We attempted reversing the order in preliminary experiments but NSI I love being put in the hospital for dehydration. In future, we would like to generate more diverse sarcasm that are not tied to a fixed pattern. Finally, the generations are dependent on COMET and thus the quality will be governed by the accuracy of the COMET model. We address the problem of unsupervised sarcasm generation that models several sarcasm factors including reversal of valence and semantic incongruity with the context. The key contribution of our approach is the modeling of commonsense knowledge in a retrieve-and-edit generation framework. A human-based evaluation based on four criteria shows that our generation approach significantly outperforms a state-of-the-art model. Compared with human generated sarcasm, our model shows promise particularly for creativity, humor and sarcasticness, but less for grammaticality. A bigger challenge in sarcasm generation and more generally, creative text generation, is to capture the difference between creativity (novel but well-formed material) and nonsense (ill-formed material). Language models conflate the two, so developing methods that are nuanced enough to recognize this difference is key to future progress. This work was supported in part by the MCS program under Cooperative Agreement N66001-19-2-4032 and the CwC program under Contract W911NF-15-1-0543 with the US Defense Advanced Research Projects Agency (DARPA). The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. The authors would like to thank Christopher Hidey, John Kropf, Anusha Bala and Christopher Robert Kedzie for useful discussions. The authors also thank members of PLUSLab at the University Of Southern California and the anonymous reviewers for helpful comments.
objective method result result abstain abstain other abstain other method method objective objective objective objective objective abstain abstain objective objective method result objective method method result objective abstain abstain abstain abstain abstain method abstain abstain abstain objective abstain abstain abstain abstain abstain abstain method abstain method abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method method method other abstain method abstain abstain abstain abstain abstain abstain method method method method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method other method abstain abstain abstain other abstain abstain result result result abstain abstain abstain abstain result abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method abstain abstain method abstain method abstain abstain abstain abstain method abstain method abstain method abstain abstain objective result result abstain abstain other other other other