Previous sarcasm generation research has focused on how to generate text that people perceive as sarcastic to create more human-like interactions. In this paper, we argue that we should first turn our attention to the question of when sarcasm should be generated, finding that humans consider sarcastic responses inappropriate to many input utterances. Next, we use a theory-driven framework for generating sarcastic responses, which allows us to control the linguistic devices included during generation. For each device, we investigate how much humans associate it with sarcasm, finding that pragmatic insincerity and emotional markers are devices crucial for making sarcasm recognisable. The prevalence of sarcasm on the social web (Kho-dak et al., 2018; Sykora et al., 2020) has motivated computational investigations across the NLP community. Most focus on textual sarcasm detection, the task of classifying whether or not a given text is sarcastic (Riloff et al., 2013; Wallace et al., 2015; Rajadesingan et al., 2015; Bamman and Smith, 2015; Joshi et al., 2016; Amir et al., 2016; Hazarika et al., 2018; Oprea and Magdy, 2019; Abu Farha et al., 2022). A recent research direction considers sarcasm generation. Approaches to sarcasm generation introduced so far (Joshi et al., 2015; Mishra et al., 2019; Chakrabarty et al., 2020) are mainly motivated by the potential to create more approachable, human-like conversational agents, considering that sarcasm is a natural part of human discourse. We suggest reconsidering this motivation, as a community, for two reasons. First, in human discourse, sarcasm is not a communicative goal in itself. Rather, it can be used to achieve a wide variety of goals. Some of these goals, such as to diminish the impact of criticism (Dews and Winner, 1995), to create humour (Kreuz et al., 1991; Colston and O'Brien, 2000b,a), to praise (Bruntsch and Ruch, 2017), or to strengthen relationships (Jorgensen, 1996; Pexman and Zvaigzne, 2004), might be desirable in human-machine interactions as well. However, other goals, such as criticising, mocking, or expressing dissociation, often with surface contempt or derogation (Wilson, 2006), might not be desirable in human-machine interactions. Second, the communicative goals mentioned above were observed in human interactions. Even when a machine seeks potentially desirable goals, it is unclear whether sarcastic utterances have the same effect on humans when coming from machines. RQ1. When should a chatbot be sarcastic? (a) When do humans consider sarcasm appropriate? (b) When do humans prefer sarcasm, over non-sarcasm? RQ2. How should a chatbot formulate sarcasm? (a) What linguistic devices do humans associate with sarcasm? (b) What sarcasm flavour do they prefer? Here, by flavour , we mean a specific conjunction of linguistic devices that humans may associate with sarcasm, such as intensifiers and emotional markers, as introduced in Section 3, and expanded upon in Section 4. To address our research questions, we suggest the following approach. First, given a set of input utterances, generate several sarcastic responses. Each response should be of a specific sarcasm flavour, i.e. should display a specific conjunction of linguistic devices. Next, create a survey that asks human participants: to indicate how appropriate it was to respond sarcastically to the input; to select their preferred response; and to rate the sarcasticness of each response, investigating whether they associate the linguistic devices in the response with sarcasm. To achieve this, we require a sarcastic response generator that provides control over the linguistic devices used. Most previous generators rely on variants of the traditional theory of sarcasm, which claims that the intended meaning concealed by sarcasm is the opposite of the literal meaning. However, this theory provides a grounding that is neither necessary, nor sufficient, for sarcasm to occur, as discussed in Section 3. To overcome this limitation, we recently introduced Chandler, a novel modular sarcastic response generation framework (Oprea et al., 2021). It is grounded on a formal theory that, from a linguistic-theoretical perspective, specifies devices whose presence is both necessary and sufficient to unambiguously differentiate sarcasm from non-sarcasm. These are allusion to a failed expectation, pragmatic insincerity, and emotional markers. Chandler can generate sarcasm of different flavours, and allows control over flavour its output should reflect. Herein, we also compare Chandler's outputs to those of previous generators, to examine participant preferences toward an even greater range of sarcasm flavours. Our results indicate that people find sarcastic responses inappropriate for most input utterances. When sarcasm was considered appropriate, the inputs commonly had a positive sentiment, and often had elements of humour. Further, even when considered appropriate, people still did not usually prefer sarcastic responses over non-sarcastic ones. Sarcasm was typically preferred when it was also considered funny and not too specific. Finally, we identified pragmatic insincerity and emotional markers (cf. Section 3) as crucial linguistic devices to include in generating recognizable sarcasm. Contributions We summarise our contributions as follows. First, our approach allows us to understand people's preferences about when sarcasm should be used, and how it should be formulated. Using this information, we provide guidelines for future work in sarcasm generation. Second, observing people's preferences also allows us to quantitatively evaluate the practical advantages of the formal linguistic theory that grounds Chandler. The earliest work on sarcasm generation is that of Joshi et al. (2015), who introduce SarcasmBot, a sarcastic response generation system. SarcasmBot uses one of eight possible generators, each containing a set of predefined patterns, one of which is instantiated as the response. The generators do not in fact account for the meaning of the input, rather, they only focus on aspects such as the overall sentiment or presence of swear words. Further, in our experiments, we noticed that most of the time a fallback generator was employed, returning the simple concatenation of a random positive phrase to a random negative one, from a set of predefined phrases that have no specific connection to the input. Mishra et al. (2019) suggest a sarcastic paraphrase generator. They assume that the input is always of negative polarity, and suggest an unsupervised pipeline of four modules to convert such an input u (  ) to a sarcastic version. In the Sentiment Neutralisation module, they filter out negative sentiment words from u (  ) to produce u (0) . In the Positive Sentiment Induction module, they modify u (0) to convey positive sentiment, producing u (+) . Next, in the Negative Situation Retrieval module, they mine a phrase v (  ) that expresses a negative situation. v (  ) is selected from a set of predefined phrases, based on the similarity to the original input. Finally, the Sarcasm Synthesis module constructs the sarcastic paraphrase from u (+) and v (  ) . Chakrabarty et al. (2020) suggest a similar pipeline. Their R 3 system first employs a Reversal of Valence module, which replaces input words of negative valence with their lexical antonyms using WordNet (Miller, 1995) to produce u (+) . Next, it builds an utterance v that is incongruous to u (+) , and generates sarcasm from u (+) and v . Previous generators share a limitation that make them unfit for our purposes. Mainly, relying on the traditional theory, they identify sarcasm with linguistic incongruity. Thus, they only provide this single device for investigation, device that is not sufficient for sarcasm to occur, as discussed in Section 3. A further limitation, shared by Mishra et al. (2019) and Chakrabarty et al. (2020), is that their generators only work with input utterances of negative sentiment. However, as discussed earlier, sarcastic communication can have many goals, 7687 including to praise, or to strengthen friendships. In this section, we describe the Implicit Display Theory, a formal linguistic theory that grounds Chandler. Previous Theories In the traditional theories , sarcasm is created by literally saying one thing but figuratively meaning, or conversationally implicating (Grice, 1975), the opposite. However, such incongruity is not necessary for sarcasm. To see this, consider sarcastic understatements such as saying This was not the best movie ever to mean the movie was bad. It is also not sufficient. For instance, it also occurs in the construction of certain stylistic devices, such as metaphors, e.g. Time is money. Further theories have been suggested to address these limitations, including the echoic mention theory (Sperber and Wilson, 1981) and its variants (Kreuz and Glucksberg, 1989; Wilson and Sperber, 1992; Sperber and Wilson, 1998), and the pretense theory (Clark and Gerrig, 1984) and its variants (Clark, 1996). However they all fail to uniquely identify sarcasm, as argued by Utsumi (2000) and Oprea and Magdy (2020). Implicit Display Theory (IDT) Introduced by Utsumi (1996), the IDT focuses specifically on making the distinction between sarcasm and non-sarcasm. We invite the interested reader to consult (Utsumi, 2000) for an overview of how it overcomes the limitations of previous theories. We chose it as a grounding for our generation system. The IDT first defines the concept of an ironic environment. We say a situation in which an utterance occurs is surrounded by an ironic environment if the discourse context includes the following components: (1) The speaker has expectation Q at time t 0 ; (2) Q fails at time t 1 > t 0 ; and (3) The speaker has a negative attitude towards the failure of Q . Note that the idea of linking sarcasm to an expectation is not new to Utsumi (1996), rather it is supported by previous work (Kreuz and Glucksberg, 1989; Kumon-Nakamura et al., 1995). Next, according to the IDT, an utterance is sarcastic if and only if it implicitly displays the ironic environment. Implicit display is realised if the following linguistic devices are present in the utterance: (1) allusion to the speaker's failed expectation Q ; (2) pragmatic insincerity, realised by intentionally violating one of the pragmatic principles, e.g. Grice's maxims (Grice, 1975); and (3) implication (indirect expression) of the speaker's negative attitude towards the failure of Q . Finally, the theory claims that the degree of sarcasm of an utterance is proportional to how many of these linguistic devices are present in the utterance. In this section we look at the methodology employed to address our research questions. Specifically, we first select a set of input utterances. Next, for each input, we generate four sarcastic responses of different flavours using Chandler, and three more responses using other systems. Finally, for each input, in a survey, we ask human participants to rate the responses across several dimensions, to understand their preference towards the appropriateness of sarcasm, and which linguistic devices they associate with sarcasm. As inputs, we select texts from the corpus published by Wilson and Mihalcea (2019). The corpus contains short texts (extracted from tweets) where users describe actions they performed. We compute the sentiment polarity of each text using the classifier from Barbieri et al. (2020), a RoBERTa model (Liu et al., 2019) fine-tuned on the tweet sentiment dataset from Rosenthal et al. (2017). Next, we form five partitions of 50 texts each: very negative and very positive , containing the top 50 texts based on their negative and positive probabilities, respectively; negative , containing random texts for which the probability of being negative was higher that the probabilities of being positive or neutral; and positive and neutral , partitions that we formed analogously to how we formed the negative partition. Our final input dataset contains 250 texts. For completeness, in this section we describe Chandler, the sarcastic response generator that we introduced in Oprea et al. (2021). The IDT directly suggests an algorithm for sarcasm generation that identifies an ironic environment, then creates an utterance that implicitly displays it. We now discuss how we implement each step. Ironic Environment As discussed in Section 4.1, each input text U in describes an action. In this scenario, herein, we assume the expectation 7688 Q that is part of the ironic environment negates that action. For instance, say U in expresses the event P = [ <user> wins the marathon ] . We assume Q =  P = [ <user> does not win the marathon ] . As we shall see, the algorithm we suggest will not, in fact, require us to formulate Q, but it relies on the above assumption. Allusion to Q Following Utsumi (2000), we de-fine allusion in terms of coherence relations, similar to the relations of rhetorical structure theory (RST) (Mann and Thompson, 1987). That is, if U  is an utterance that expresses proposition  , we say U  alludes to the expectation Q if and only if there is a chain of coherence relations from  to Q 1 . So, we need to first select a proposition  to either start or end the coherence chain, then specify the chain between  and Q , and formulate U  such that it expresses  . We suggest defining such  as objects of if-then relations, where the subject is P , the proposition expressed by input text U in . That is, relations of the form if P then   should hold. To infer  given U in , we use COMET (Bosselut et al., 2019), an adaptation framework for constructing commonsense knowledge. Specifically, we use the COMET variant fine-tuned on ATOMIC (Sap et al., 2019), a dataset of typed if-then relations. COMET inputs the subject of the relation, along with the relation type, and outputs the relation object. In our case, the subject is U in , and we set  to the relation object. In the examples that follow, assume the input text is U in = <user> won the marathon'. We leverage four relation types: (1) xNeed : the object  of a relation of this type specifies an action that the user needed to perform before the event took place, e.g. if U in then  = [ xNeed to train hard ] ; (2) xAttr : the object  specifies how a user that would perform such an action is seen, e.g. if P then  = [ xAttr competitive ] ; (3) xReact : the object  specifies how the user could feel as a result of the event, e.g. if P then  = [ xReact happy ] ; and (4) xEffect : the object specifies a possible effect that the action has on the user, e.g. if P then  = [ xEffect gets congratulated ] . In Table 1 we show, for each relation type, the coherence chains between the relation object  and the failed expectation Q . Under these conditions, to generate an utterance U  that alludes to Q , we need to choose 1 Note that a restriction in Utsumi (2000)'s definition of allusion is that U does not directly express the state of affairs that Q is expected via phrases such as "I've expected ...". Pragmatic insincerity The second requirement for implicit display is that the utterance generated should include pragmatic insincerity. In this paper, we focus on violating Grice's maxim of quality (Grice, 1975), where we aim for the propositional content of the generated utterance to be incongruous to that of U in (input text). To achieve this, we first choose an if-then relation type, then infer the relation object  from U in using COMET, and construct an utterance that expresses   . For instance, if U in = <user> won the marathon', and we have chosen the xAttr relation type, the constructed utterance could express   = [ <user> is not competitive ] . Negative attitude To fulfill the last requirement of implicit display, the utterance generated should imply a negative attitude towards the failure of the expectation Q . As pointed out by Utsumi (1996), this can be achieved by embedding verbal cues usually associated with such attitudes, including hyperbole and interjections. Logical form and explainability At this point we formulate Algorithm 1 for generating a sarcastic response U out , given an input utterance U in that expresses proposition P . We refer to emotion (   ) as the logical form of the sarcastic response we generate. Here, emotion is a function that augments   to express a negative attitude. Note that the logical form, together with the coherence chain between  and the failed expectation Q , provide a complete explanation for how and why sarcasm occurs. The explanation is (cid:15) = ( emotion (   ) , C ) , where is C the coherence chain from  to Q . The coherence chain for each relation type can be selected from Table 1. This makes our sarcasm generation process accountable. Logical Form to Text To convert the logical form to text, we rely on predefined patterns for each if-then relation type. As a running example, 7689 relation type example relation coherence chain xNeed if P then  = [ xNeed to train hard ] volitional-cause ( , P ) and contrast ( P, Q ) xAttr if P then  = [ xAttr competitive ] condition ( , IP )  purpose ( IP , P )  contrast ( P, Q ) xReact if P then  = [ xReact happy ] contrast ( Q, P )  volitional-result ( P,  ) xEffect if P then  = [ xEffect gets congratulated ] contrast ( Q, P )  non-volitional-result ( P,  ) Table 1: Coherence chains between the object  of an if-then relation and the failed expectation Q , for each relation type, as discussed in Section 4.2. assume the input utterance U in = <user> won the marathon' and the chosen relation type is xAttr . Say  = COMET ( U in , xAttr ) = [ xAttr competitive ] . The logical form is emotion (  [ xAttr competitive ]) . We first construct an intermediate utterance U  using the following rule: <user> <verb> competitive . Here, <verb> is a verb specific to each relation type. In our example, U  could be <user> is competitive'. Next, for each input U in , we generate three responses. The first response U  e out only includes pragmatic insincerity, i.e. it expresses  [ xAttr competitive ] . To construct it, we apply a rule-based algorithm to generate the negation of U  in a manner similar to (Chakrabarty et al., 2020), discussed in Section 2. U  e out could be <user> is not competitive'. The second response U  i out does not include pragmatic insincerity, but only markers that express an emotional attitude, i.e. it expresses emotion ([ xAttr competitive ]) . To achieve this, in a pattern-based manner, we augment U  with hyperbole and interjections, as indicated by Utsumi (2000). U  i out could be <user> is definitely competitive, yay!'. The third response U out includes both devices, i.e. it expresses emotion (  [ xAttr competitive ]) . U out could be <user> is definitely not competitive, yay!'. A full list of patterns is shown in Section A in the appendix. In the running example we focused on the xAttr relation type. Recall there are four relation types that we consider, xNeed , xAttr , xReact , and xEffect . As such, for each input text U in , we generate 12 responses: three response types, U  e out , U  i out , and U out , for each relation type. We use the pattern Ch-<relation > ( | i | e )? to refer to each response of our system, Chandler . For instance, Ch-xAttr refers to U out built considering the xAttr relation, while Ch-xNeed  e refers to U  e out built considering the xNeed relation. Note that other strategies for converting the logical form of sarcasm to text are possible. For instance, using policy-based generation with external rewards (Mishra et al., 2019) might have lead to higher perceived sarcasticness of our generated responses. However, we leave this to future work. Our goal is to understand user preferences towards when sarcasm should be used, and how sarcasm should be formulated. We built three surveys, labelled (a)(c), that we published on the Prolific Academic 2 crowdsourc-ing platform, one for each output type, out of U  e out , U  i out , and U out . As such, in the survey corresponding to U out , we presented participants with the input text U in , along with the responses produced by Chandler-xNeed, Chandler-xAttr, Chandler-xReact, and Chandler-xEffect. In each survey, we also enclosed a response from DialoGPT (Zhang et al., 2020), a recent dialogue system that is not built to be sarcastic; a response produced by SarcasmBot, the sarcastic response generator of Joshi et al. (2015) ; and a response produced by R 3 , the state-of-the-art sarcastic paraphrase generator of Chakrabarty et al. (2020) 3 . We make a few observations. First, DialoGPT is used as a reference system, following the reasoning of Joshi et al. (2015): responses designed to be sarcastic should have a higher perceived sarcasticness than responses from DialoGPT, which are not designed to be sarcastic. Second, note that R 3 is designed to produce rephrases. As such, we applied R 3 to the output of DialoGPT to get a sarcastic rephrase of a response to the input. Table 2 shows an example input utterance, along with responses from all systems. All in all, each survey instance contained a specific input text, and seven responses generated as mentioned above and presented in a random order. In the survey, we asked participants to evaluate each response across four dimensions: (1) Sarcasm: How sarcastic is the response? (2) Humour: How 2 https://prolific.co 3 https://github.com/tuhinjubcse/SarcasmGeneration-ACL2020 7690 system response DialoGPT I'm not sure if you're being sarcastic or not. funny is the response? (3) Coherence: How coherent is the response to the input? It is coherent if it sounds like sensible response that a person might give in a real conversation; and (4) Specificity: How specific is the response to the input? It is not specific if it can be used as a response to many other inputs. Each dimension ranged from 0 to 4, in line with previous work (Chakrabarty et al., 2020). Next, we asked participants to select their preferred response out of the seven, i.e. the one they would personally use. Finally, we asked them to judge, on a scale from 0 to 4, how appropriate it was to respond sarcastically to the shown input text. Each survey instance was presented to three different participants. However, we did not use a voting scheme to aggregate the three survey instances into one. Rather, aggregation was conducted per-system. This is because our metrics (e.g. sarcasticness, preference towards a response, appropriateness) are inherently subjective, depending on the sociocultural background of the participants. See, for instance, the work of Oprea and Magdy (2020). As such, the concept of correct answer does not exist in the conventional sense. Indeed, the inter-participant agreement was low, but not surprisingly so, given that participants could have come from different sociocultural backgrounds. However, this does not entail that population statistics are not informative. As related work in this direction, consider that of Amidei et al. (2018), who make the very pos pos neutral neg very neg 0 1 2 Figure 1: Mean sarcasm appropriateness score for each sentiment category, as discussed in Section 5.1. point an unchecked focus on reduction of disagreement among annotators runs the danger of creating generation goals that reward output that is more distant from, rather than closer to, natural humanlike language. (Amidei et al., 2018) Consider also the work of Davani et al. (2021), who discuss the issue of disagreement in subjective tasks. We do, however, encourage more work in this direction. We now look at the responses that the participants provided in our survey, addressing our RQs. Figure 1 shows the mean appropriateness score for each of the five sentiment categories. A one-way ANOVA test between the means yielded a p -value  0 . 001 . We therefore proceeded with Tukey's range test (Tukey, 1949), to find the means that are significantly different from one another. We noticed that sarcasm was considered significantly more appropriate by survey participants in responses to positive inputs, compared to very positive, and very negative inputs, respectively. This supports our statement from Section 2: the assump-tion of previous state-of-the-art generators that sarcasm should only be generated for negative inputs is problematic. However, even for the positive class, the mean appropriateness is less than 2. This makes it difficult to recommend responding sarcastically based on sentiment only. To gain more insight, we proceeded with a qualitative inspection of the inputs that yielded the highest and lowest appropriateness scores, respectively. We noticed a few main themes, that we labelled joke , family , school , leisure and death . We then asked two humans to label all inputs across these dimensions. A third human resolved all disagreements. Finally, we computed the Pearson correlation coefficient of each theme with the sarcasm appropriateness score, across all inputs. We noticed a significant ( p < 0 . 05 ) positive correlation 7691 text approp. between appropriateness and the category joke , and significant negative correlation with belonging to the family theme. We show some examples of the theme family with low appropriateness scores in Table 3. Thus, according to our analysis, sarcasm seems to be most appropriate for positive inputs, and for humorous inputs, which may invite more sarcastic responses. In other situations, however, sarcasm might be interpreted as inappropriate and even offensive (Meaney et al., 2021). We first consider the overall preference towards either sarcasm or non-sarcasm. Recall that participants also specified their preferred response for each input. The distribution of the sarcasm, humour, specificity, and coherence scores of this preferred response, across all survey instances, is illustrated in Figure 2 with a blue, continuous, line. The red, dashed, line illustrates the distribution across the 80 survey instances where the sarcasm appropriateness score of the input was higher than the midpoint, i.e. at least 3. We notice considerably higher preference towards non-sarcastic and non-humorous responses. As indicated by the blue lines, over 50% of the preferred responses were those considered non-sarcastic and non-humorous by participants, the rest of the distribution being highly skewed towards the lower sarcasm and humour regions. Furthermore, note that even when sarcasm was considered highly appropriate, participants still preferred non-sarcastic responses, as indicated by the red, dashed, line in the top-left of Figure 2. Although there is a shift in the distribution towards sarcasm in this case, the skew is still towards the non-sarcastic region. Looking at the bottom row of Figure 2, on the other hand, we notice a negative skew, indicating an overall preference towards higher coherence. This is slightly the case for specificity as well. To investigate further, we fit a logistic regression model to predict whether a response is preferred based on its sarcasm, humour, specificity, coherence scores, and two-way interactions between these variables. All coefficients are listed in Appendix B. We noticed noticed a significant ( p < 0 . 05 ) positive relationship between coherence and preference, as well as the interaction between sarcasm and humour. The term representing the product of sarcasm and specificity had a significant negative effect on preference. In terms of the specific systems, we notice DialoGPT was preferred about 44% of the time, followed by Ch-xAttr  i (20%), and SarcasmBot (15%), which corresponds exactly to the coherence ranking in Table 4. Our results indicate that responses with high coherence to the inputs are generally preferred over sarcastic responses. Sarcasm is only preferred when it is also considered humorous. On the other hand, participants seem to have actively avoided sarcastic responses that were very specific. In Table 4 we show mean sarcasm, humour, specificity, and coherence scores provided by participants for each variant of Chandler, across all inputs. In the table, there are four groups (14) and three systems within each group (ac). Rows with index (a) show scores for the complete versions of Chandler, for each if-then relation type. Rows (b) and (c) show partial versions, omitting pragmatic insincerity and emotional markers, respectively. Allusion We have four strategies for alluding to the failed expectation, depending on the relation type considered. We notice the highest sarcasm score is achieved by Ch-xAttr (row 2a), followed by Ch-xNeed (row 1a), Ch-xReact (row 3a) and Ch-xEffect (row 4a). The same ranking holds for 7692 System sarc. variants of Chandler that do not include pragmatic insincerity or emotional markers. Out of the allusion strategies selected, the responses perceived as most sarcastic are those that mention attributes of the user. Similarly, we notice that variants of Chandler that use the xAttr relation are also perceived and the most coherent, specific to the input, and achieve the highest humour score. Pragmatic Insincerity Comparing the complete version, Ch-xAttr (row 2a), with Ch-xAttr  i (row 2b), the same model without pragmatic insincerity, we notice a significant drop in average sarcasm score. We observe a similar trend in group 3 for Ch-xReact  i , indicating the importance of pragmatic insincerity. However, this did not hold for the other two relation types. Additionally, both specificity and coherence seem to significantly increase when removing pragmatic insincerity, irrespective of the relation type considered. Emotional Markers Comparing complete versions of Chandler with those that omit emotional markers, we notice that the omission of such markers leads to significantly lower perceived sarcasm for all relation types. Humour is also significantly impacted by the omission of emotional markers for all relation types considered except for xEffect (row 4). Oh the other hand, coherence and specificity are not significantly influenced. To sum up, the degree of perceived sarcasm is influenced by all linguistic devices considered. Out of the if-then relation types we consider, mentioning attributes of the user seems to lead to the highest perceived sarcasm, humour, specificity and co-very pos pos neutral neg very neg 0 0.2 0.4 0.6 DialoGPT SarcasmBot DialoGPT+R3 Ch-xNeed Ch-xAttr Ch-xReact Figure 3: Normalized number of times each system was preferred for instances were the participant preferred a response that they also considered sarcastic. herence. Being insincere about the state of affairs leads to significantly higher perceived sarcasm, but significantly lower specificity and coherence. Emotional markers increase sarcasm and humour perception, but do not significantly impact specificity or coherence. Finally, recall that a main claim of IDT was that the degree of sarcasticness of an utterance grows with the number of implicit display conditions met. Our results support this claim. While we established that participants typically preferred non-sarcatic responses, we next set out to find what sarcasm people preferred in our experiments when they did prefer sarcasm. To do this, we consider the set of survey instances that showed the complete versions of Chandler, where the sarcasm score given by the participant to their preferred response was at least 3, leaving us with 107 (around 14%) of the 750 survey instances. We divide these instances into five categories, based on input sentiment. Within each category, for each generation system, we count the number of times that a response produced by that system was preferred. Figure 3 shows the normalised counts across all systems, for each sentiment category. We observe that, for positive inputs, where sarcasm was considered significantly more appropriate than other sentiment categories, people prefer responses produced by Ch-xNeed. Interestingly, however, we observe that people prefer the fairly nonspecific, pattern-based sarcastic remarks produced by SarcasmBot for most types of input text. However, when analysing its outputs, we noticed it produced a total of only 28 unique responses (listed in Appendix C) to our 250 inputs. While in our experiments each response was only shown at most three times, in a real scenario of a user interacting with a conversational agent, the user might not appreciate repeatedly receiving the same response. We recommend that future work on sarcasm generation should account for the four main findings: (1) People think sarcasm is inappropriate as a response to most inputs. However, if it is to be used, it is seen as most appropriate when the input is positive, but not extremely positive. People also found sarcasm to be a suitable response to jokes. (2) Even when deemed appropriate, people usually do not prefer sarcasm. Rather, coherence is the most important factor in explaining their response preferences. When people do prefer sarcasm, they like it mainly when it is also seen funny. Further, they generally dislike sarcasm that is very specific. (3) When generating sarcasm, pragmatic insincerity and emotional markers are important to include as they have a high influence of sarcasm perception. (4) Overall, people commonly prefer the simple general sarcastic responses of SarcasmBot, even compared to more sophisticated generation models, which suggests that presently, a simpler solution to sarcasm generation may actually be advantageous. Nevertheless, more investigation is required to examine if it will be desirable in long conversations, since it has limited diversity in outputs. We have used a linguistically informed framework for sarcasm generation so that we could present human judges with a variety of flavors of sarcastic responses in a range of situations. Our findings suggest that sarcasm should not always be generated, but the decision to generate sarcasm itself should informed by user preferences. People find sarcasm most appropriate as a response to positive utterances and cases in which a joking environment has already been established. Further, judges preferred sarcasm most when they actually found it to be funny, and most often preferred general sarcastic responses. However, people often preferred non-sarcastic responses even more. We recommend that future work in this area carefully considers both the appropriateness and necessity of generating sarcasm at all. In our experiments, we noticed that some of the input tweets contained references to sensitive top-ics, such as religion and gender, or to tragic life events. Producing sarcasm for such inputs might be inappropriate and offensive to some (as our experiments confirmed). We clearly informed our survey participants about this possibility in the Participant Information Sheet, before accessing our survey. The sheet is enclosed in Appendix D. References Ibrahim Abu Farha, Silviu Vlad Oprea, Steven R. Wilson, and Walid Magdy.
abstain objective method objective abstain other abstain abstain objective abstain abstain abstain abstain abstain abstain objective objective objective objective objective objective objective objective method abstain abstain abstain result abstain abstain abstain abstain abstain method result abstain abstain abstain method objective objective method result other other other method other other other other other other other other other other objective other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain method abstain method abstain abstain abstain method method abstain abstain abstain method abstain method abstain method abstain method abstain abstain method abstain method method abstain abstain abstain other abstain method abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain method other abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain method abstain abstain method abstain method method abstain abstain abstain method abstain abstain method other abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result result abstain abstain abstain result result abstain result method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain method abstain abstain