Abstract This paper addresses the problem of dialogue reasoning with contextualized commonsense inference. We curate CICERO , a dataset of dyadic conversations with five types of utterance-level reasoning-based inferences: cause, subsequent event, prerequisite, motivation, and emotional reaction. The dataset contains 53,105 of such inferences from 5,672 dialogues. We use this dataset to solve relevant generative and discriminative tasks: generation of cause and subsequent event; generation of prerequisite, motivation, and lis-tener's emotional reaction; and selection of plausible alternatives. Our results ascertain the value of such dialogue-centric commonsense knowledge datasets. It is our hope that CICERO will open new research avenues into commonsense-based dialogue reasoning. Conversational content on the internet is quickly growing, and such content holds valuable knowledge about how information exchange takes place among speakers. A key step towards understanding such dialogues is gaining the ability to reason with the information shared in the dialogue. To this end, we curate a dataset of dyadic conversations named CICERO ( C ontextual I zed C ommons E nse Infe R ence in dial O gues) , which contains inferences around the utterances in the dialogues. The dataset focuses on five types of reasoning-based inferences for a given utterance in a dialogue: cause, subsequent event, prerequisite, motivation, and emotional reaction. Arguably, making such reasoning-based inferences often demands commonsense knowledge, especially when the inference is implicit. Fig. 1a shows such a case where the cause behind the target utterance is not explicit in the context. allowed the annotator to infer a probable cause of the utterance. On the other hand, commonsense can be crucial in sifting relevant information from the context. Fig. 1b depicts an instance where the cause behind the target utterance is inferred from the context. This inference can be explained by commonsense knowledge (see Fig. 3) such as repetitive consumption of the same food causes  boredom dispelled by  changing food achieved by  eating at McDonald's . Thus, it is reasonable to posit that such knowledge could aid to bridge the gap between the input and the target inference. ATOMIC (Sap et al., 2019; Hwang et al., 2020) is one such dataset for commonsense reasoning-based inference, allowing for a large set of inference types. However, ATOMIC is context-free, as it only provides inferences on short phrases, ignoring the broader context around them. Making an inference on an entire utterance, on the other hand, requires understanding the context around it. As per Grice's maxim (Grice, 1975), in conversations, the interlocutors provide any piece of 5010 information as is needed, and no more. Thus, much of the information required to understand an utterance is likely interspersed along the dialogue, and not necessarily localized in the given utterance. For instance, in the example in Figure 1b, understanding the cause for one of the speakers' desire to go to McDonald's requires the context of the previous utterances. ATOMIC is thus not ideal for commonsense reasoning-based inferences on dialogues, where context is critical for understanding an utterance's implications. We confirm this with our experiments in the subsequent sections (4). GLUCOSE (Mostafazadeh et al., 2020) exclusively curates causal inferences  cause, enable, and result in  from monologues. Thus, it is not ideal for making context-consonant inferences on the dialogues. Also, dialogue-specific dimensions like motivation and reaction are beyond its scope. On the other hand, CIDER (Ghosal et al., 2021a) does provide a dataset for commonsense-based inference on dialogues, but it is limited to inferences explicitly observable in the dialogues. As such, sys-tems based on CIDER cannot e  ectively speculate around the dialogue for implicit inference. CICERO strives to bring the best of these three datasets by creating a dataset that can enable models to e  ectively operate on a dialogue by considering the context and speculating when the answer is not apparent. We create CICERO  a large dataset of English dyadic conversations annotated with five types of inferences with the help of human annotators, who are instructed with a carefully crafted set of guidelines. The annotators are given a dialogue and a target utterance, as exemplified in Fig. 2. The annotators are then asked to make an inference, posed as a question, about the target utterance. They write a one-sentence answer that is grammatically correct, concise, and consistent with the dialogue. The answer may contain both overt and speculative scenarios. An overt scenario is explicitly or implicitly present in the dialogue context. If such contextual scenarios answer the question, the annotators write them as a well-formed sentence. However, in many cases, the dialogue may not hold the answer, neither explicitly nor implicitly. In such cases, the annotators are asked to speculate plausible scenar-Linda would you care for some candies or cookies? ios around the dialogue, using commonsense and world knowledge, to devise answers that do not contradict the given dialogue context. Given the dialogue-target pair in Fig. 2, at least one of the following five inferences about the target is made by the annotators: Dad, what will we have for dinner? Q1. What is the event that directly causes (overt) or could cause (speculative) Target ? The annotators consider if any of the events that are or likely to be antecedent to the target can cause the target . Answer: Linda didn't exercise regularly during the winter. Remark: The annotators provided possible, speculative answers as the dialogue itself does not provide any reason for Linda's weight gain. Q2. What subsequent event happens (overt) or could happen (speculative) following the Target ? The annotators write about the event that happens or could happen following the target . Additionally, annotators were told that sometimes, such subsequent events of the target are triggered or likely to be triggered by the target . Answer: Linda starts a diet and tries to lose weight. Q3. What is (overt) or could be (speculative) the prerequisite of Target ? Does the target have any direct prerequisite or dependency that has to happen or be fulfilled first? (In most cases, prerequisite is the state / event which has to be satisfied before another event causes target .) The answer is a state / event which enables the happening of the target . In other words, prerequisites are the prior assumptions or background information that the interlocutors agree on about the context. Answer: Linda was slimmer before the winter. Remark: Annotators were required to understand the di  erence between cause and prerequisite clearly before proceeding with the final annotation. Cause of an event X is the event that directly causes X. Prerequisite of an event X is the condition which has to be satisfied in order for X to happen. Q4. What is an emotion or basic human drive that motivates or could motivate Target ? Consider the basic human drives, needs (and / or likely emotions) of the speaker of the target . Basic human drives and needs are food, water, clothing, warmth, rest, security, safety, intimate relationships, friends, prestige, feeling of accomplishment, self-fulfillment, creative activities, enjoyment, etc. Do any of these human drives / states of mind / emotional feelings motivate the target ? Answer: Not Applicable for this target. the listener: A (or B)? What could be the possible emotional reaction or responses of the listener with respect to the target ? The annotators capture the appropriate emotion of the listener using the emotion terms listed in Table 1 verbatim or related words (e.g., anxious, confused, interested, etc). Answer: The listener encourages Linda to maintain her diet. annotators to adhere to the following guidelines: Be creative in speculation. Refrain from rephrasing the target and writing low-e  ort trivial answers. It is recommended to skip a question if rephrasing the target is the only possible answer. Avoid repeating the same answer for distinct questions on the same target . The answer must be consistent with the given dialogue. It is recommended to base the answer on the most important phrase of the target should it contain multiple phrases. DailyDialog (Li et al., 2017) covers dialogues from wide range of topics  life, work, relationships, tourism, finance, etc. The constituent utterances are labelled with emotion and dialogue-act. MuTual (Cui et al., 2020) is a multi-turn dialogue reasoning dataset. Given a dialogue history, the objective is to predict the next utterance by considering aspects such as intent, attitude, algebraic, multi-fact, and situation reasoning. DREAM (Sun et al., 2019) is a multiple-choice reading-comprehension dataset collected from exams of English as a foreign language. The dataset presents significant challenges as many answers are non-extractive and require commonsense knowledge and multi-sentence reasoning. 1. We remove dialogues that are too short or long on either utterance or word level. Dialogues with fewer than five utterances or fewer than six words per utterance on average are removed. Dialogues having more than 15 utterances or more than 275 words in total are also removed. 2. All three source datasets contain dialogues having near identical utterances. We remove these near duplicate dialogues to ensure topical diversity of CICERO . We use a sentence embedding model based on fine-tuned RoBERTa (Gao et al., 2021) to extract dense feature vectors of the dialogues. We remove the duplicates assuming that a pair of duplicate dialogues have at least 0 . 87 cosine similarity. We first determine the number of target utterances in D : if D has 16 utterances, then we select 2 or 3 targets; if it has 712 utterances then we select 35 targets; otherwise, we select 47 targets if it has more than 12 utterances. We divide D into 23 segments having roughly equal number of consecutive utterances. We choose roughly an equal number of the top-ranking utterances from each segment. We call this set of utterances x 1 . The ranking is performed using a sentence ranking algorithm (Erkan and Radev, 2004; Mihalcea and Tarau, 2004) with sentence-BERT embeddings (Reimers and Gurevych, 2019a). We also select the longest utterances in D and the utterances that contain phrases such as I'm, I'd, I've, I'll or their expansions. We call this set of utterances x 2 . The sets x 1 and x 2 may not be disjoint. Set x 3 consisting of the final utterance of D . From x 1  x 2 : Subsequent Event: 80% of the targets. Both Cause and Prerequisite: 60% of the targets. Exclusively Cause: 28% of the targets. Exclusively Prerequisite: 12% of the targets. From x 2 : Motivation for all targets. Initially, we sample 50 random dialogues and manually annotate all the questions (as in 2.1) in those. Each annotator is then evaluated on those dialogues, and is selected for the annotation task if 95% of his / her annotations are approved by us. We constantly review and provide feedback to the annotators during the annotation process. Annotators are also instructed to amend their answers. Upon completion of the annotation, we employ three additional annotators who manually check the annotated samples and score their acceptability. These annotators reached a consensus for approving 86% of these samples. The samples not bearing majority agreement were removed from the dataset. A ( u 1 ) ( u 1 ) ( u 1 ) : Hi, Jenny. Is it true you're moving to London? B ( u 2 ) ( u 2 ) ( u 2 ) : Yes, it is. A ( u 3 ) ( u 3 ) ( u 3 ) : What made you decide to do that? B ( u 4 ) ( u 4 ) ( u 4 ) : Work, mainly. I'm sure I'll be able to find a job there. A ( u 5 ) ( u 5 ) ( u 5 ) : You're probably right. But where are you going to live? B ( u 6 ) ( u 6 ) ( u 6 ) : I hope I'll find a flat to share with somebody. That way it will be cheaper. A ( u 7 ) ( u 7 ) ( u 7 ) : Yes, that's a good idea. Are you taking your dog with you? B ( u 8 ) ( u 8 ) ( u 8 ) : No, I don't think so. My parents have o  ered to take care of him, and I don't think he'd be happy in the city. A ( u 9 ) ( u 9 ) ( u 9 ) : You're probably right. But aren't you afraid of moving to such a big place, especially after living in a small village? B ( u 10 ) ( u 10 ) ( u 10 ) : Not really. I think I'll enjoy myself. There's so much to do there; I expect I won't miss the countryside much and I can always come back and visit. A ( u 11 ) ( u 11 ) ( u 11 ) : Well, I just hope you'll invite me to stay when you get settled. B ( u 12 ) ( u 12 ) ( u 12 ) : Of course I will. Target u 6 u 6 u 6 ; Inference: Cause ; Annotation: Being an expensive city, it is quite di  cult to find an a  ordable place to live in London. Target u 10 u 10 u 10 ; Inference: Cause ; Annotation: Jinny realizes that a city like London will provide a great quality of life for her. Target u 6 u 6 u 6 ; Inference: Subsequent Event ; Annotation: The listener gives an idea to Jenny to find the flat on some online portal for searching flatmates as well plenty of cheaper options. Target u 10 u 10 u 10 ; Inference: Subsequent Event ; Annotation: Jenny inquired a social club in London and ask for their membership to utilize her free time. Target u 4 u 4 u 4 ; Inference: Prerequisite ; Annotation: Jenny has completed her studies. Following Table 3, a majority (  59%) of the inferences in CICERO are causal in nature. Again, roughly 80% of the inferences are speculative and context consonant. CICERO is thus much more versatile in terms of its applications as compared to CIDER (Ghosal et al., 2021a) that only contains explicit contextual inferences. CICERO also contains varied commonsense knowledge  from general to physical and social commonsense (see Appendix B for more details). We design generative and multi-choice question answering tasks on CICERO to evaluate dialogue-level commonsense-based reasoning capabilities of The objective is to generate the answer to question q , representing one of the five inference types, for a target utterance u t in a dialogue D . Each inference type has its respective q (illustrated in 4). Task 1.1: Dialogue Causal Inference. Causality pertains to causes and e  ects of events and situations. We formulate the dialogue causal inference task as generating the cause or subsequent event of an utterance as an answer to a causal question: 1. Cause: Given D , u t , generate the cause c t of u t . 2. Subsequent Event: Given D , u t , generate the subsequent event e t of u t . 3. Subsequent Event Clipped (Subsequent EC): Given u t , the dialogue up to u t : D : u t , generate the subsequent event e t of u t . We consider two di  erent scenarios for subsequent event , as the event often appear after the target utterance in the dialogue. Hence, subtask 3 is more challenging to evaluate a models' ability to reason about unobserved e  ects. We extend subtasks 1, 2 to incorporate longer chains and formulate the chained generation task. We consider utterances u t in our dataset that has both cause and subsequent event annotated i.e. c t  u t  e t . The causal chain is considered as a triplet, and we formulate tasks where a missing segment has to be generated from the rest of the components: 4. Chained Cause : Generate c t from u t and e t . 5. Chained Subsequent Event (Chained SE) : Generate e t from u t and c t . prerequisite / motivation / reaction of listener from a given D and u t . The target u t is the final utterance of D for reaction generation. Generating the prerequisite (task 1.2.1) requires an understanding of the dependency of events. Generating the motivation (task 1.2.2) and reaction (task 1.2.3) is about learning basic human drives and emotions. Note that, reaction generation is a di  erent problem from dialogue response generation. Responses follow utterance level distributions which are substantially di  erent from emotional reactions. Given dialogue D , target u t , one of the five questions (inference type) q , true answer a t , alternate choices F t = { f t 1 , f t 2 , f t 3 , f t 4 } , the CICEROMCQ task aims to select the correct answer a t (see Fig. 4) and additionally any answer among F t which might be correct. The alternate choices F t are created through a combination of automated generation and human supervision as follows:  We train a T5 large model on SNLI contradictory pairs (Bowman et al., 2015) and Time-Travel counterfactual pairs (Qin et al., 2019) to generate contradictions / counterfactuals from input sentences. We use this model to generate a pool of alternate answers from the true annotated answers. Alternate answers which have an embedding cosine similarity less than 0.9 with the true answer (from all-mpnet-base-v2 in Reimers and Gurevych (2019b)) and are contradictory w.r.t the true answer (from roberta-large-mnli ) are kept, and the rest are discarded. The filtered set is termed N . We use the adversarial filtering (AF) algorithm (Zellers et al., 2018) to select the four alternate answers F t from N . For multi-choice QA tasks, AF is an e  ective method to detect easily identifi-able alternate answers and replace them with more di  cult candidates by detecting and reducing stylistic artifacts. The algorithm is as follows: (i) We start with annotated true answer a t and any four choices  F t from N for all instances in our dataset to create  D . We randomly split  D into  D train (80%) and  D test (20%) according to dialogue IDs. (ii) A multi-choice QA model (discriminator) is trained on  D train that scores all five choices for all instances in  D test . The highest scoring choice is considered as the predicted answer. For a particular test instance, choices in  F t that have lower scores than a t are replaced with other high scoring choices in N   F t . Answers in  F t which are being replaced 5014 A: Can I help you? (iii)  F t now consists of relatively more di  cult choices. A new random split  D train and  D test is created, and we go back to step (ii) . The algorithm is terminated when the accuracy in successive  D test reaches a convergence. The final alternate choice set is termed as F t . The AF algorithm ensures a robust final dataset D irrespective of the final train, validation, and test split. We use a new roberta-large model to initialize the discriminator and train for 3 epochs before scoring and replacement in step (ii) . 14 iterations were required for convergence in D test . Annotators perform manual checking on the final AF selected choices F t . They mark each of the alternate choices in F t in D to be speculatively correct or incorrect given the context. Hence, instances might have correct answers in F t in addition to the originally annotated correct answer a t . The final dataset statistics after this step are given in Table 3. Task 2.1: Single Answer Selection. Consider instances where F t doesn't contain any correct answer. The task is to select the correct answer a t among the five choices given D , u t , and q . Task 2.2: All Answers Selection. This task is performed on the entire dataset (including the subset of data which is used in Task 2.1. There might be one or more correct answers for a particular instance resulting from the AF algorithm. The task is to select all the correct answer(s) (including a t ) among the five choices given D , u t , and q . We split our dataset in dialogue level where the training, validation and test instances are obtained from a total of 3477, 1097, 1098 distinct dialogues respectively. This results in a 60:20:20 proportion of total annotation instances. The three sets have 17365, 5370, and 5331 unique target utterances respectively. We tune on the validation dataset and report results on the test dataset (average of 5 runs). For the sake of brevity, the detailed hyperparame-ters are given in the supplementary material. We use the following questions ( q ) for the five inference types for all the tasks: Cause : What is or could be the cause of target? Subsequent Event : What subsequent event happens or could happen following the target? Prerequisite : What is or could be the prerequisite of target? Motivation : What is or could be the motivation of target? Reaction : What is the possible emotional reaction of the listener in response to target? CICERONLG  (1.11.2). We use large versions of T5 (Ra  el et al., 2020) and GLUCOSE-T5 (Mostafazadeh et al., 2020) as our models. GLUCOSE-T5 is a T5 large model that is pre-trained on the GLUCOSE dataset. We concatenate q , u t , and the context c with separators to form the input to the model: q <sep> u t <sep> c . The context c is formed by concatenating utterances of D : u t (subsequent event clipped) or D (all other tasks). For the chained generation task, we additionally provide the cause / subsequent event as input. The inputs are q <sep> u t <sep> subsequent event: e t <sep> c and q <sep> u t <sep> cause: c t <sep> c for cause and subsequent event generation, respectively. The objective is to generate the answer as output in the sequence-to-sequence setup. We use teacher forcing during training and beam search during inference. CICEROMCQ  Single Answer Selection (2.1). We use RoBERTa-large , ELECTRA-large , T5-large , and Unified QA Large for this task. The input to the models for RoBERTa-large , ELECTRA-large is the concatenation of question q , target u t , dialogue D , and candidate answers x j , j  { 1 , ..., 5 } : <cls> q <sep> u t <sep> D <sep> x j . Each score is predicted from the corresponding <cls> vector and the highest scoring one is selected as the answer. For seq2seq 5015 models T5-large , and Unified QA Large , we use the following input  q <sep> 1) x 1 2) x 2 3) x 3 4) x 4 5) x 5 <sep> u t <sep> D . The output to be generated is the correct answer  such as x 1 or x 2 . CICEROMCQ  All Answers Selection (2.2). We use seq2seq models T5-large , and Unified QA Large as they can generate both single and multiple-answers (with separator tokens) as output. The input is q <sep> 1) x 1 2) x 2 3) x 3 4) x 4 5) x 5 <sep> u t <sep> D . The output to be generated are the correct answer(s), such as x 2 (single answer) or x 1 <sep> x 3 <sep> x 4 (multi-ple answers). Here, x 1  x 5 denotes the five possible choices shu  ed randomly. Automatic Evaluation Metrics. For generative tasks, we report the following metrics: BLEU (Pap-ineni et al., 2002), METEOR (Banerjee and Lavie, 2005), ROUGE (Lin, 2004), CIDEr (Vedantam et al., 2015), and Sem-Sim which computes the semantic cosine similarity of two sentences using the supervised RoBERTa-large sentence embedding model (Gao et al., 2021). All scores are reported in the range of 0-1. Human Evaluation Metrics. Due to significant dissonance with human evaluation, automatic evaluation metrics are often considered not reliable for generation quality evaluation in literature. Hence, we resort to human evaluation metrics. The human annotators rate on an integer scale from 1 (worst) to 5 (best) on three coarse attributes: Creativity : As the majority of the inferences require speculation, this metric measures how creative the models and the annotators are. Contextuality : Whether the generated or annotated inferences fit the context. Fluency : Whether the generated or annotated inferences are grammatically correct. Results of Automatic Evaluation. The results for the generative tasks are reported in Table 4 and Table 5. We observe that the fine-tuned models perform quite similarly across various metrics in Table 4. The T5 model achieves the best performance in most of the experimental settings. The results indicate that the causal types are more challenging to infer than the Motivation , and Reaction . However, the models are posed to the most challenging instances in the case of Prerequisite type as inferring this type requires rich commonsense and back-Model BLEU2 METEOR ROUGE CIDEr Sem-Sim ( 1 . 1 . 1 ) C a u s e T5 0.1493 0.1630 0.2626 0.4560 0.6278 GLUCOSE-T5 0.1563 0.1634 0.2707 0.4915 0.6305 T5  0.0042 0.0200 0.0266 0.0237 0.3735 GLUCOSE-T5  0.0287 0.0560 0.0827 0.1332 0.4442 ( 1 . 1 . 2 ) SE T5 0.1619 0.1662 0.2760 0.4119 0.6276 GLUCOSE-T5 0.1611 0.1628 0.2778 0.4430 0.6297 T5  0.0045 0.0191 0.0264 0.0241 0.3865 GLUCOSE-T5  0.0001 0.0070 0.0024 0.0032 0.3073 ( 1 . 1 . 3 ) SEC li pp e d T5 0.1448 0.1549 0.2618 0.3099 0.6123 GLUCOSE-T5 0.1461 0.1523 0.2645 0.3238 0.6094 T5  0.0199 0.0439 0.0564 0.0762 0.4549 GLUCOSE-T5  0.0001 0.0066 0.0025 0.0034 0.3063 ( 1 . 2 . 1 ) P rere qu i s i t e T5 0.1002 0.1282 0.2176 0.3357 0.5902 GLUCOSE-T5 0.1001 0.1299 0.2197 0.3144 0.5896 T5  0.0043 0.0222 0.0279 0.0225 0.3541 GLUCOSE-T5  0.0108 0.0394 0.0625 0.0889 0.4392 ( 1 . 2 . 2 ) M o t i va t i o n T5 0.2503 0.1998 0.3781 0.7109 0.6973 GLUCOSE-T5 0.2582 0.2037 0.3840 0.7499 0.7048 T5  0.0033 0.0183 0.0257 0.0181 0.4038 GLUCOSE-T5  0.0174 0.0434 0.0632 0.0696 0.4053 ( 1 . 2 . 3 ) R e a c t i o n T5 0.2397 0.1939 0.3720 0.5177 0.6665 GLUCOSE-T5 0.2318 0.1903 0.3716 0.5364 0.6653 T5  0.0037 0.0201 0.0239 0.0167 0.3899 GLUCOSE-T5  0.0213 0.0459 0.0759 0.0719 0.4125 Table 4: Results of the CICERONLG task. ground knowledge. Hence, for this category, the models achieve a low score compared to rest of the inference categories. We also notice that exposing the future utterances to the models help in attaining better inference performance for the relation type Subsequent Event . The trained models perform worse when the future utterances are not available in the input as seen in the Subsequent Event Clipped task. A significant drop of performance is noticed in the CIDEr metric. For the chained generation tasks (1.1.4 and 1.1.5), we notice (refer to Table 5) a very similar trend in models' performance i.e., the models tend to perform better for these two experimental settings compared to only Cause (1.1.1) and Subsequent Event (1.1.2) predictions. We can surmise that the additional cues from the available annotations of Subsequent Event type in the Chained Cause setting, and the Cause type in the Chained Subsequent Event setting are the key 5016 Metric Gold T5 GLUCOSE T5  GLUCOSE  Creativity 4.7 3.8 3.9 2.4 1.9 Contextuality 4.8 4.1 4.3 2.1 2.1 Fluency 5.0 4.8 4.9 1.9 2.9 Table 6: Results of the human evaluation for CICERONLG . Target u 1 u 1 u 1 ; Inference: Cause ; Answers: Gold The speaker did not eat breakfast. T5 The speaker and the listener are going to a restaurant for lunch. GLUCOSE-T5 The speaker and the listener are at a restaurant. Target u 2 u 2 u 2 ; Inference: Cause ; Answers: Gold The restaurant down the street is famous for its delicious food. T5 The listener is hungry and wants to eat something. GLUCOSE-T5 The speaker is hungry and wants to order something to eat. Target u 3 u 3 u 3 ; Inference: Motivation ; Answers: Gold The speaker is craving some savory fast food. T5 The speaker is craving for a hamburger, fries and a chocolate sundae. GLUCOSE-T5 The speaker is craving for a burger, fries and sundae. Target u 6 u 6 u 6 ; Inference: Prerequisite ; Answers: Gold The speaker is a fitness freak and keeps track of his daily diet. T5 The speaker is a healthy person. GLUCOSE-T5 The speaker is a health conscious person. Target u 7 u 7 u 7 ; Inference: Subsequent Event ; Answers: Gold The listener refused to eat anything that is unhealthy. T5 The speaker and the listener decided to order some hot dogs. GLUCOSE-T5 The speaker and the listener decided to order some hot dogs. Target u 8 u 8 u 8 ; Inference: Reaction ; Answers: Gold The listener felt embarrassed by the statement of the speaker. T5 The listener is shocked to hear the speaker's comment. GLUCOSE-T5 The listener is disappointed with the speaker's decision. A ( u 1 ) ( u 1 ) ( u 1 ) : I'm hungry, let's order up something to eat. B ( u 2 ) ( u 2 ) ( u 2 ) : Ok, maybe we can order a soup and a salad from the restaurant down the street. A ( u 3 ) ( u 3 ) ( u 3 ) : I was thinking of getting a hamburger, fries and a chocolate sundae. B ( u 4 ) ( u 4 ) ( u 4 ) : You eat too much junk food. That sort of stu  clogs up your arteries and is very high in cholesterol. A ( u 5 ) ( u 5 ) ( u 5 ) : Well I never seem to gain weight so I don't mind. B ( u 6 ) ( u 6 ) ( u 6 ) : It's not only about getting fat or not, it's about being healthy. You could really have some health problems later on. A ( u 7 ) ( u 7 ) ( u 7 ) : How about pizza or maybe some fried chicken! Better yet, let's order some hot dogs! B ( u 8 ) ( u 8 ) ( u 8 ) : You are a lost cause. to such performance improvement. As depicted in Table 4 (and also Table 6), the non fine-tuned versions of T5 and GLUCOSE-T5 perform poorly as they produce gibberish outputs across all the five inference categories indicating the importance of fine-tuning on CICERO . Results of Human Evaluation. For each of the five inference types, we randomly sample 40 inferences generated by each model and their corresponding gold inferences. These inferences are then manually rated by three independent annotators based on the human-evaluated metrics. As suggested by Table 6, we observe that most of the fine-tuned models on CICERO perform similarly but fail to reach gold annotation performance. Moreover, as expected, the fine-tuned models significantly outperform their non fine-tuned counterparts. We provide some examples of the generated inferences in Table 7. Inspection of the model generated inferences reveal that usage of keywords from the dialogue without generalizing the events is more frequent. Generated inferences are significantly less diverse and creative than gold annotations. Performance of GLUCOSE. GLUCOSE contains contextual commonsense inferences on events in monologues. Comparing the results (Table 4, Table 6) of fine-tuned and non fine-tuned checkpoints suggests that pre-training on a monologue-based contextual commonsense inference dataset does not ensure good performance on the same task for dialogues. Akin to the non fine-tuned T5, non fine-tuned GLUCOSE-T5 produces gibberish outputs for all the commonsense inference types but the causal and motivation types. We surmise this happens as these two commonsense types exist in the GLUCOSE dataset. Although the generated text for these two commonsense inference types are grammatically correct and sometimes contain contextual words, they are far from the desired quality, semantically very much dissimilar from the annotated gold instances, and rated low in the qualitative evaluation, as shown in Table 6. We also confirm the e  cacy of fine-tuning the models on CICERO through human evaluation, as explained in 4. Evaluation Metrics. 1) RoBERTa and ELECTRA : The accuracy of selecting the correct answer is used to evaluate the performance of these models. 2) T5 and Unified QA : The output is considered as a single answer if it doesn't contain any separator token. Otherwise, the output is segmented at separator tokens to obtain multiple answers. We then follow the method in Khashabi et al. (2020), where match is computed by comparing each of the generated answer(s) with the candidate choices based on their token-level overlap. For each generated answer, the most similar candidate choice is considered as the corresponding output. The prediction is considered as correct if the final output(s) is an exact match (EM) with the gold annotated answer(s). Single Answer Selection (2.1). We report the results of this setting in Table 8. The reported metric is accuracy of selecting the correct answer. The overall score is 83.28% for RoBERTa and 86.82% for ELECTRA . ELECTRA has an edge over RoBERTa on all the five inference types. This could be a side e  ect of using RoBERTa as the backbone model for the AF algorithm and subsequently as a solver for 5017 Model Cause SE Prereq. the final CICEROMCQ task. We think, this results expose the model dependency of the AF process. In other words, the negative samples chosen by the backbone model X for the AF algorithm will be dif-ficult to distinguish from the human-annotated true samples using the same model X . These negative samples, however, could be relatively easier to identify using another model Y . The seq2seq models T5 and Unified QA perform significantly better than RoBERTa and ELECTRA as can be seen in Table 8. While models like RoBERTa, ELECTRA encode each candidate answer separately, T5 and Unified QA encode them together. Thanks to this joint encoding of candidate answers, T5 and Unified QA can take advantage of more task-related information that RoBERTa and ELECTRA might miss due to the separate encoding scheme. We surmise it could be one of the reasons why the seq2seq models have an edge over RoBERTa and ELECTRA for this particular task. T5 and Unified QA attain almost the same score for single answer selection. This is surprising as Unified QA is initialized from the T5-large checkpoint and then further trained on other QA datasets. As such, we think, the di  erent fine-tuned domains of Unified QA does not help in the CICEROMCQ task. All Answers Selection (2.2). We train and evaluate T5 and Unified QA on the entire dataset of both single and multiple correct answers and report the results in Table 9. Overall, T5 and Unified QA perform similarly. The general performance, across the models, on instances with multiple correct answers is much worse than instances with a single correct answer. We confirm this by reporting the results only on instances with multiple answers in Table 9, where T5 and Unified QA achieve only 3.38% and 3.60% exact match, respectively. This could probably be attributed to the stark data imbalance of  86 / 14% between singleand multi-answer instances, respectively (see Table 3). Commonsense knowledge has received more attention compared with factual knowledge, as it is usually not mentioned explicitly in the context. It is demonstrated to be essential in open-ended generation tasks, such as story explanation generation (Mostafazadeh et al., 2020), story end generation (Guan et al., 2019) and abductive reasoning (Bhagavatula et al., 2019). To infuse commonsense knowledge in NLP models, several approaches to tasks like sentence ordering (Ghosal et al., 2021b), emotion recognition (Ghosal et al., 2020), story generation (Guan et al., 2020; Xu et al., 2020) and dialogue generation (Zhou et al., 2018) use prevalent commonsense knowledge bases (CSKB) like ConceptNet (Speer et al., 2017) or ATOMIC (Sap et al., 2019). However, ConceptNet is context-free, meaning that they only capture relationships around a selected set of entities, without paying attention to the context where the entity occurs. Moreover, inference is often needed in discourse level, which do not always align with the entities in knowledge bases. Knowledge models such as COMET (Bosselut et al., 2019) is a way to circumvent this issue and make inferences on an utterance (sentence) level. But the generated knowledge still lacks the detail from the dialogue, as it is trained on the aforementioned knowledge base. Our approach, instead, centers on the dialogue dataset and provides more detailed commonsense inference at an utterance level. We introduced CICERO , a new dataset for dialogue reasoning with contextualized commonsense inference. It contains  53K inferences for five commonsense dimensions  cause, subsequent event, prerequisite, motivation, and emotional reaction  collected from  5.6K dialogues. To show the usefulness of CICERO for dialogue reasoning, we design several challenging generative and multi-choice answer selection tasks for state-of-the-art NLP models to solve. This work is supported by the A*STAR under its RIE 2020 AME programmatic grant RGAST2003 and project T2MOE2008 awarded by Singapore's MoE under its Tier-2 grant scheme. The annotators for CICERO were hired through a data annotation service. The compensation was derived based on the country of residence of the annotators, as deemed by the company. The study has been categorized as exempt by the IRB. Annotators were strictly asked not to write any toxic content (hateful or o  ensive toward any gender, race, sex, religion). They were asked to consider gender-neutral settings in dialogues whenever pos-sible.The source dialogue datasets  DailyDialog, MuTual, and DREAM are high quality multi-turn dialogue datasets manually annotated by experts in dialogue, communication theory and linguistics. All three datasets have been extensively used and studied in the natural language processing literature. The three source datasets and our annotations in CICERO do not contain any personal data or any information that can uniquely identify individual people or groups.
abstain abstain objective abstain objective result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method method abstain objective method method abstain abstain abstain abstain abstain abstain abstain objective objective objective objective objective method method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain method abstain method abstain abstain method abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain other abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain other other other other other other other method objective abstain objective other abstain abstain abstain abstain abstain abstain method