Abductive reasoning aims at inferring the most plausible explanation for observed events, which would play critical roles in various NLP applications, such as reading comprehension and question answering. To facilitate this task, a narrative text based abductive reasoning task  NLI is proposed, together with explorations about building reasoning framework using pretrained language models. However, abundant event commonsense knowledge is not well exploited for this task. To fill this gap, we propose a variational autoencoder based model ege-RoBERTa, which employs a latent variable to capture the necessary commonsense knowledge from event graph for guiding the abductive reasoning task. Experimental results show that through learning the external event graph knowledge, our approach outperforms the baseline methods on the  NLI task. Abductive reasoning aims at seeking for the best explanations for incomplete observations (Bhagavat-ula et al., 2019). For example, given observations Forgot to close window when leaving home and The room was in a mess , human beings can generate a reasonable hypothesis for explaining the observations, such as A thief entered the room based on commonsense knowledge in their mind. However, due to the lack of commonsense knowledge and effective reasoning mechanism, this is still a challenging problem for today's cognitive intelligent systems (Charniak and Shimony, 1990; Oh et al., 2013; Kruengkrai et al., 2017). Most previous works focus on conducting abductive reasoning based on formal logic (Eshghi et al., 1988; Levesque, 1989; Ng et al., 1990; Paul, 1993). However, the rigidity of formal logic limits the application of abductive reasoning in NLP  Corresponding author Figure 1: tasks, as it is hard to express the complex semantics of natural language in a formal logic system. To facilitate this, Bhagavatula et al. (2019) proposed a natural language based abductive reasoning task  NLI. As shown in Figure 1 (a), given two observed events O 1 and O 2 , the  NLI task requires the prediction model to choose a more reasonable explanation from two candidate hypothesis events H 1 and H 2 . Both observed events and hypothesis events are daily-life events, and are described in natural language. Together with the  NLI task, Bhagavatula et al. (2019) also explored conducting such reasoning using pretrained language models such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019). However, despite pretrained language models could capture rich linguistic knowledge benefit for understanding the semantics of events, additional commonsense knowledge is still necessary for the abductive reasoning. For example, as illustrated in Figure 1 (b), given observations O 1 and O 2 , to choose the more likely explanation H 1 : A thief entered the room and exclude H 2 : A breeze blew in the window , prediction model should have the commonsense knowledge that it is hardly possible for a breeze to mess up the room, whereas a thief may enter the room from the open window ( I 1 ), then rummage through the room ( I 2 ) and lead to a mess. These intermediary events ( I 1 and I 2 ) can serve as necessary commonsense knowledge for understanding the relationship between observed events and hypothesis events. We notice that the observed events, hypothesis events, intermediary events and their relationships could be described using an event graph, which can be constructed based on an auxiliary dataset. The challenge is how to learn such commonsense knowledge from the constructed event graph. To address this issue, we propose an Event Graph Enhanced RoBERTa (ege-RoBERTa) model, and a two-stage training procedure. Specifically, as shown in Figure 1 (c), on the basis of the RoBERTa framework, we additionally introduce a latent variable z to model the information about the intermediary events. In the pretraining stage, ege-RoBERTa is trained upon an event-graph-based pseudo instance set to capture the commonsense knowledge using the latent variable z . In the finetuning stage, model adapts the commonsense knowledge captured by z to conduct the abductive reasoning. Experimental results show that ege-RoBERTa could effectively learn the commonsense knowledge from a well-designed event graph, and improve the model performance on the  NLI task compared to the baseline methods. The code is released at https://github.com/sjcfr/ege-RoBERTa. As shown in Figure 1 (a),  NLI can be defined as a multiple-choice task. Given two observed events O 1 and O 2 happened in a sequential order, one needs to choose a more reasonable hypothesis event from two candidates H 1 and H 2 for explaining the observations. Therefore, we formalize the abductive reasoning task as a conditional distribution p ( Y | O 1 , H i , O 2 ) , where H i  { H 1 , H 2 } , and Y  [0 , 1] is a relatedness score measuring the reasonableness of H i . In the  NLI dataset, H i is set to be an explanation event happens intermediate to O 1 and O 2 (Bhagavatula et al., 2019). Hence, O 1 , O 2 and H i form an event temporal sequence O 1 , H i , O 2 . For brevity, we denote the event sequence as X = ( O 1 , H i , O 2 ) . Therefore, taking the event order into consideration, we further characterize the abductive reasoning task as p ( Y | X ) . Formally, an event graph could be denoted as G = { V, R } , where V is the node set, and R is the edge set. Each node V i  V corresponds to an event, while R ij  R is a directed edge V i  V j along with a weight W ij , which denotes the probability that V j is the subsequent event of V i . Given observed events and a certain hypothesis event, from the event graph we could acquire additional commonsense knowledge about: (1) the intermediary events, (2) the relationships between events . As Figure 1 (b) shows, the observed events, hypothesis event and intermediary events compose another event sequence ( O 1 , I 1 , H i , I 2 , O 2 ) . For clarity, we define such event sequence as posterior event sequence X (cid:48) , where X (cid:48) = ( O 1 , I 1 , H i , I 2 , O 2 ) . The relationship between events within X (cid:48) could be described by an adjacency matrix A  R 5  5 , with each element initialized using the edge weights of the event graph: A jk = (cid:26) W jk , if V j  V k  R, 0 , others . The matrix A could describe the adjacency relationship between arbitrary two events in X (cid:48) . 3 Ege-RoBERTa as a Conditional Variational Autoencoder Based Reasoning Framework In this paper, rather than directly predicts the relatedness score Y based on the event sequence X , we propose to predict Y based on both X and additional commonsense knowledge (i.e. posterior event sequence X (cid:48) and adjacency matrix A ). To this end, we introduce a latent variable z to learn such knowledge from an event graph through a two stage training procedure. To effectively capture the event graph knowledge through z and conduct the abductive reasoning task based on z , we frame the ege-RoBERTa model as a conditional variational autoencoder (CVAE) (Sohn et al., 2015). Specifically, with regard to the latent variable z , ege-RoBERTa characterizes the conditional distribution P ( Y | X ) using three neural networks: a prior network p  ( z | X ) , a recognition network q  ( z | X (cid:48) , A ) and a neural likelihood p  ( Y | X, z ) , Figure 2: Illustration of the pretraining, finetuning and prediction process of ege-RoBERTa. where  and  denote the parameters of networks. Moreover, instead of directly maximize P ( Y | X ) , following CVAE (Sohn et al., 2015), ege-RoBERTa proposes to maximize the evidence lower bound (ELBO) of P ( Y | X ) : LELBO ( ,  ) = E q  ( z | X (cid:48) ,A ) log( p  ( Y | X, z ))  KL( q  ( z | X (cid:48) , A ) || p  ( z | X ))  log p ( Y | X ) (2) Note that, in the recognition network, the latent variable z is directly conditioned on X (cid:48) and A , where X (cid:48) = { O 1 , I 1 , H i , I 2 , O 2 } is the posterior event sequence, A is an adjacency matrix describing the relationship between events within X (cid:48) . This enables z to capture the event graph knowledge from X (cid:48) and A . Through minimizing the KL term of ELBO, we can teach the prior network p  ( z | X ) to learn the event graph knowledge from the recognition network as much as possible. Then in the neural likelihood p  ( Y | X, z ) the relatedness score Y could be predicted based on X and z , which captures the event graph knowledge. However, the event graph knowledge is absent in the  NLI dataset. To learn such knowledge, we design the following two-stage training procedure: Pre-training Stage: Learning Event Graph Knowledge from a Pseudo Instance Set In this stage, ege-RoBERTa is pretrained on a prebuilt event-graph-based pseudo instance set, which contains rich information about the intermediary events and the events relationships. As shown in Figure 2 (a), the latent variable z is directly conditioned on X (cid:48) and A . Therefore, z could be employed to learn the event graph knowledge. Finetuning Stage: Adapt Event Graph Knowledge to the Abductive Reasoning Task As Figure 2 (b) shows, at the finetuning stage, ege-RoBERTa is trained on the  NLI dataset Figure 3: Architecture of ege-RoBERTa. without the additional information X (cid:48) and A . In this stage model learns to adapt the captured event graph knowledge to the abductive reasoning task. Then as Figure 2 (c) shows, after the two-stage training process, ege-RoBERTa could predict the relatedness score Y based on the latent variable z . We introduce the specific implementation of ege-RoBERTa. As illustrated in Figure 3, ege-RoBERTa introduces four modules in addition to the RoBERTa framework: (1) an aggregator providing representation for any event within X and X (cid:48) ; (2) an attention-based prior network for modeling p  ( z | X ) ; (3) a graph neural network based recognition network for modeling q  ( z | X (cid:48) , A ) ; (4) a merger to merge the latent variable z into RoBERTa frame for downstream abductive reasoning task. The event representation aggregator provides distributed representation for events in both the event sequence X and the posterior event sequence X (cid:48) . To this end, the aggregator employs attention mechanism to aggregate token representations of the event sequence from hidden states of RoBERTa. Given an event sequence X composed of tokens [ [CLS], ( x 11 ,..., x 1 l 1 ),...,( x 31 ,..., x 3 l 3 ) ] (where [CLS] is the special classification token (Devlin et al., 2019), and x jk is the k th token within the j th event), the M th transformer layer of RoBERTa encodes these tokens into contextualized distributed representations H ( M ) = [ h [ CLS ] , ( h 11 ,..., h 1 l 1 ),...,( h 31 ,..., h 3 l 3 ) ] , where h jk  R 1  d is the distributed representation of the k th token within the j th event. Then for the j th event, the distributed representation is initialized as q j = 1 l j (cid:80) h jl j . Multi-head attention mechanism ( MultiAttn ) (Vaswani et al., 2017) is employed to softly select information from H ( M ) and get the representation of each event: e j = MultiAttn( q j , H ( M ) ) . For brevity, we denote the vector representation of all events in X using a matrix EX , where EX = { e 1 , e 2 , e 3 }  R 3  d . Note that, through the embedding layer of RoBERTa, position information has been injected into the token representations. Therefore, EX derived from token representations carries event order information. In addition, since EX is obtained from the hidden states of RoBERTa, rich linguistic knowledge within RoBERTa could be utilized to enhance the comprehension of event semantics. By the same way, the representation of events within X (cid:48) could be calculated, which we denote as EX (cid:48) . The recognition network models q  ( z | X (cid:48) , A ) based on EX (cid:48) and A , where EX (cid:48) is the representations of events within X (cid:48) . Following traditional VAE, q  ( z | X (cid:48) , A ) is assumed to be a multivariate Gaussian distribution: q  ( z | X (cid:48) , A )  N (  (cid:48) ( X (cid:48) , A ) , D ) , (4) where D denotes the identity matrix. To obtain  ( X (cid:48) , A ) , we first combine EX (cid:48) and adjacency matrix A using a GNN (Kipf et al., 2016): E ( U ) (cid:48) =  ( AEX (cid:48) W ( u ) ) . (5) where  (  ) is the sigmoid function; W ( u )  R d  d is a weight matrix and E ( U ) (cid:48) are relational information updated event representations. Then a multi-head self-attention operation is performed to promote the fusion of event semantic information and relational information: E ( U ) (cid:48) = MultiAttn( E ( U ) (cid:48) , E ( U ) (cid:48) ) . (6) Finally, to estimate  ( X (cid:48) , A ) , we aggregate information within E ( U ) (cid:48) using a readout function g (  ) :  (cid:48) = g ( E ( U ) (cid:48) ) . (7) Following Zhou et al. (2019) and Zhong et al. (2019), we set g (  ) to be a mean-pooling operation. Hence, by estimating  (cid:48) based on the relational information updated event representation E ( U ) (cid:48) , event graph knowledge about X (cid:48) and A is involved into the latent variable z . The prior network models p  ( z | X ) based on EX , where EX is the representation matrix of events in X . The same as the recognition network, p  ( z | X ) also follows multivariate normal distribution, while the parameters are different: p  ( z | X )  N (  ( X ) , D ) , (8) where D denotes the identity matrix. To obtain  ( X ) , different from the recognition network, the prior network starts from updating EX using a multi-head self-attention: E ( U ) = MultiAttn( EX , EX ) . (9) Then an additional multi-head self-attention operation is performed to get deeper representations: E ( U ) = MultiAttn( E ( U ) , E ( U ) ) . (10) Finally,  ( X ) is estimated through aggregating information from E ( U ) :  = g ( E ( U ) ) , (11) where g (  ) is a mean-pooling operation. The merger module merges the latent variable z as well as updated (deep) representation of events into the N th transformer layer of RoBERTa frame for predicting the relatedness score. To this end, we employ multi-head attention mechanism to softly select relevant information from z and E ( U ) , and then update the hidden state of the N th transformer layer of RoBERTa. Note that, given X , p  (  | X ) achieves its maximum when z =  . Hence, making predictions based on  could be regarded as finding the best explanation based on the most likely commonsense situation. Through integrating latent variable z , H ( N )  contains the event graph knowledge. By taking H ( N )  as the input of the subsequent ( N + 1) th transformer layers of RoBERTa for predicting the relatedness score, the abductive reasoning task is conducted based on the additional event graph knowledge. The  NLI task requires model to choose a more likely hypothesis event from two candidates. However, in the pre-training stage, the negative examples are absent in the pseudo instances. To address this issue, following the method of Liu et al. (2019), in the pre-training stage ege-RoBERTa is trained to predict the masked tokens in the event sequence X rather than the relatedness score. In addition, in order to balance the masked token prediction loss with the KL term, we introduce an additional hy-perparameter  . Hence, the objective function in the pretraining stage is defined as follows: LELBO ( ,  ) = E q ( z | X (cid:48) ,A ) log LMLM ( X, z ;  )   KL( q  ( z | X (cid:48) , A ) || p  ( z | X )) , (14) where log LMLM ( X, z ;  ) is the masked token prediction loss. Intuitively, through minimizing the KL term, we aim to transmit the event graph knowledge from the recognition network to the prior network. In the finetuning stage, ege-RoBERTa is trained to adapt the learned event graph knowledge to the abductive reasoning task. Without the recogniton network, we formulate the objective function as: L (  ) = p  ( Y | z, X ) = p  ( Y | z, X ) p  ( z | X ) . We implement two different sizes of ege-RoBERTa model (i.e. ege-RoBERTa-base and ege-RoBERTa-large) based on RoBERTa-base framework and RoBERTa-large framework, respectively. For the ege-RoBERTa-base model, in the aggregator, the prior network, the recognition network and the merger, the dimension of the attention mechanism d is set as 768, and all multi-head attention layers contain 12 heads. While for the ege-RoBERTa-large model, d is equal to 1024 and all multi-head attention layers contain 16 heads. In the ege-RoBERTa-base model, token representations are aggregated from the 7th transformer layer of RoBERTa, and the latent variable is merged to the 10th transformer layer of RoBERTa. While for the ege-RoBERTa-large model, the aggregator and merger layer are set as the 14th and 20th layer, respectively. The balance coefficient  equals 0.01. More details are provided in the Appendix. quadruples in training, development and test set, respectively. The observation events are collected from a short story corpus ROCstory (Mostafazadeh et al., 2016), while all of hypothesis events are independently generated through crowdsourcing. The event graph serves as an external knowledge base to provide information about the relationship between observation events and intermediary events. To this end, we build the event graph based on an auxiliary dataset, which are composed of two short story corpora independent to  NLI, i.e., VIST (Huang et al., 2016), and TimeTravel (Qin et al., 2019). Both VIST and TimeTravel are composed of five-sentences short stories. Totally there are 121,326 stories in the auxiliary dataset. To construct the event graph, we define each sentence in the auxiliary dataset as a node in the event graph. To get the edge weight W ij between two nodes V i and V j (i.e., the probability that V j is the subsequent event of V i ), we finetune a RoBERTa-large model through a next sentence prediction task. Specifically, we define adjacent sentence pairs in the story text (for example, [1 st , 2 nd ] sentence, [4 th , 5 th ] sentence of a story) as positive instances, define nonadjacent sentence pairs or sentences pairs in reverse order (such as [1 st , 3 rd ] sentence, [5 th , 4 th ] sentence of a story) as negative instances. After that we sample 300,000 positive and 300,000 negative instances from the auxiliary dataset. Then given an event pair ( V i , V j ), the finetuned RoBERTa-large model would be able to predict the probability that V j is the subsequent event of V i . Event Graph Based Pseudo Instance Set for Pretraining ege-RoBERTa To effectively utilize the event graph knowledge, we induce a set of pseudo instances for pretraining the ege-RoBERTa model. Specifically, given a five-sentence-story within the auxiliary dataset, as Table 1 shows, we define the 1 st and 5 th sentence of the story as two observed events, the 3 rd sentence as the hypothesis event, the 2 nd and 4 th sentence as intermediary events, respectively. In this way, the posterior event sequence X (cid:48) and the event sequence X of a pseudo instance could be obtained. In addition, given X (cid:48) , we initialize the elements of the adjacency matrix A using the edge weights of the event graph, and scale A so that its row sums equal to 1. After the above operations, each pseudo instance is composed of an event sequence X , a posterior event sequence X (cid:48) which contains intermediary event information, and an adjacency matrix A which describes relationships between events within X (cid:48) . We compare ege-RoBERTa with: GPT (Radford et al., 2018) is a multilayer-transformer based unidirectional pretrained language model. BERT (Devlin et al., 2019) is a multilayer-transformer based bi-directional pretrained language model. RoBERTa (Liu et al., 2019) refers robustly optimized BERT. SVM uses features about length, overlap and sentiment to predict the more likely hypothesis event. Infersent (Conneau et al., 2017) represents sentences using a Bi-LSTM, and predicts the relatedness score using MLP. ege-RoBERTa u(npretrained) refers to the ege-RoBERTa model without the pretraining stage. ege-RoBERTa  =0 refers to setting the balance coefficient to 0 in the pretraining stage. Note that all pretrained-language-model-based baselines (i.e., GPT, BERT and RoBERTa) are finetuned on the  NLI dataset as the method of Bhagavatula et al. (2019) to adapt to the abductive reasoning task. In addition, we also list two concurrent works: (i) L 2 R (Zhu et al., 2020) learns to rank the candidate hypotheses with a novel scoring function. (ii) RoBERTa-GPT-MHKA (Paul et al., 2020) enhances pretrained language model with social and causal commonsense knowledge for  NLI task. We list the prediction accuracy (%) in Table 2, and observe that: (1) Compared with SVM and Infersent, pretrained language model based methods: GPT, BERT, RoBERTa and ege-RoBERTa show sig-nificant better performances in abductive reasoning task. This is because through the pre-training Methods Accu. stage language models could capture rich linguistic knowledge that is helpful for understanding the semantics of events. (2) Comparison between ege-RoBERTa-large u with ege-RoBERTa-large shows that the pretraining process can increase the accuracy of abductive reasoning. In addition, comparison between ege-RoBERTa-large  =0 with ege-RoBERTa-large indicates that in the pre-training process, ege-RoBERTa could capture the event graph knowledge through the latent variable to enhance the abductive reasoning. Furthermore, the relative close performance between ege-RoBERTa-large u and ege-RoBERTa-large  =0 suggest that the main improvements of the performance is brought by the event graph knowledge. (3) Compared to RoBERTa, ege-RoBERTa achieves higher prediction accuracy for both the base and large sized model. This result confirms our motivation that learning event graph knowledge could be helpful for the abductive reasoning task. (4) According to Bhagavatula et al. (2019), human performance on the test set of  NLI is 91.4%. While the RoBERTa-large model has achieved an accuracy of 83.9%. Therefore, further improvements over RoBERTa-large could be challenging. Through learning the event graph knowledge, our proposed method ege-RoBERTa further improves the relative accuracy. (5) Our approach has comparable performance with the SOTA concurrent work, which combines RoBERTa with GPT, and incorporates social and causal commonsense into model. The combination of both methods would further increase the model performance. All studies are conducted on the development set of the  NLI using the ege-RoBERTa-base model. Influence of the Balance Coefficient In the pretraining stage, the balance coefficient  controls the trade off between event graph knowledge learning and abductive reasoning. To investigate the specific influence of the balance coefficient, we compare the performance of ege-RoBERTa model pretrained with different  . As shown in Figure 4, the prediction accuracy continues to increase as  increases from 0 to 0.01. This is because adequate event graph knowledge can offer guidance for the abductive reasoning task. While when  exceeds 0.05, the accuracy start to decrease, as the over-emphasis of event graph knowledge learning would in turn undermine the model performance. Influence of the External Commonsense Knowledge We study the specific effect of the event relational information and the intermediary event information by controlling the generation of pseudo instances. In specific, we eliminate the influence of the adjacency matrix A by replacing A with a randomly initialized matrix  A . Similarly, the influence of the intermediary events I 1 and I 2 is eliminated through substituting them by two randomly sampled events  I 1 and  I 2 . As Table 3 shows, both the replacement of A and { I 1 , I 2 } lead to obvious decrease of model performance. This demonstrates that ege-RoBERTa can use both two kinds of event graph knowledge for enhancing the abductive reasoning task. To find out if the improvement of Ege-RoBERTa is brought by a certain dataset, and the specific relationship between the model performance with the number of pseudo instances, we conduct following experiments: (1) excluding a certain dataset when inducing pseudo instances; (2) pretraining the ege-RoBERTa-base model with different number of pseudo instances. The corresponding results on the dev set of  NLI is shown in Table 4. We can find that, the elimination of both dataset leads to decrease of model performances. This suggests that the ege-RoBERTa model could capture relevant event graph knowledge from both dataset. While the prediction accuracy continues to increase along with the number of pseudo instances used for pretraining the ege-RoBERTa model. This is because the accumulation of commonsense knowledge is helpful for the abductive reasoning task. In addition, it also indicates that the model performance could be further improved if the auxiliary dataset is even more enlarged. Table 5 provides an example of model prediction results. Given two observed events O 1  hates Fall  and O 2  didn't have to experience Fall in Guam , the hypothesis event H 1  moved to Guam  is more likely to explain the two motivations of observed events. However, H 1 implicitly relies on a precondition that in Guam, Fall could be eluded. Correspondingly, in the auxiliary dataset, there is information supporting the hypothesis event H 1 that there is no Fall in Guam. In this case, ege-RoBERTa chooses the hypothesis event H 1 , whereas RoBERTa chooses the wrong hypothesis event H 2 . This indicates that ege-RoBERTa could learn the event graph knowledge in the pretraining process for improving the reasoning performance. In this paper, to involve the event graph knowledge, we formalize the posterior event sequence as X (cid:48) = { O 1 , I 1 , H i , I 2 , O 2 } . While our approach also allows other forms of posterior event sequences, such as X (cid:48) = { O 1 , H i , I 1 , O 2 } , X (cid:48) = { O 1 , I 1 , H i , O 2 } , or X (cid:48) = { O 1 , I 1 , I 2 , H i , O 2 } , etc. We also pretrained ege-RoBERTa on pseudo-instance sets derived by these manners. The results are shown in Table 6. We find that whatever forms of posterior event sequences involved in ege-RoBERTa, our approach can achieve consistently better performance than the baseline method. This confirms that our approach is sufficiently generalizable to deal with various forms of external event-sequence knowledge. Furthermore, ege-RoBERTa can also be equipped with more types of event graph knowledge, such as background knowledge by: formalizing the posterior event sequence as X (cid:48) = { B 1 , . . . , B m , E 1 , . . . , E n } , where { B 1 , . . . , B m } is a set of background events for a given prior event sequence { E 1 , . . . , E n } . This demonstrates the potential of ege-RoBERTa in learning different kinds of event graph knowledge for different event inference tasks. Most previous studies focus on formal logic based abductive reasoning (Eshghi et al., 1988; Levesque, 1989; Konolige, 1990; Paul, 1993). To infer the most reasonable hypothesis, the abductive reasoning process could be divided into two steps: (1) proposing reasonable hypotheses; (2) finding the best explanation from the hypotheses (Levesque, 1989; Konolige, 1990; Paul, 1993). However, the rigidity of formal logic limits its application in NLP domain. To facilitate this, Bhagavatula et al. (2019) proposed a text based abductive reasoning task  NLI. To solve the this task, Zhu et al. (2020) formalize  NLI as a rank learning task, and propose a novel ranking function. While Paul et al. (2020) enhances the reasoning model with social commonsense and causal commonsense knowledge. Compared to their works, for enhancing the abductive reasoning process, we propose to incorporate event graph knowledge by a CVAE based model ege-RoBERTa. In addition, we argue that our approach can be easily extended to other event inference tasks. Understanding events and their relationships are crucial for various natural language inference (NLI) tasks (Kruengkrai et al., 2017). Hence, a number of previous studies explore conducting NLI tasks based on event graphs. For example, to predict the subsequent event for a given event context, Li et al. (2018) build an event evolutionary graph (EEG), and make prediction using a scaled graph neural network. While Wu et al. (2019) predict the propagation of news event through combining an historical event propagation graph with temporal point process. In addition to the event prediction related tasks, Liu et al. (2017) propose to enhance the news recommendation by incorporating additional event graph information. Liu et al. (2016) detect the textual contradiction by using event graphs as additional evidence. In this paper, we employ event graph knowledge for guiding the abductive reasoning. To this end, we propose a variational autoencoder based framework ege-RoBERTa, which employs a latent variable z to implicitly capture the necessary event graph knowledge and enhance the pretrained language model RoBERTa. In this paper, we propose a variational autoencoder based framework ege-RoBERTa with a two-stage training procedure for the abductive reasoning task. In the pretraining stage, ege-RoBERTa is able to learn commonsense knowledge from an event graph through the latent variable, then in the following stage the learned event graph knowledge can be adapted to the abductive reasoning task. Experimental results show improvement over the baselines on the  NLI task. We thank the anonymous reviewers for their constructive comments, and gratefully acknowledge the support of the National Key Research and Development Program of China (2020AAA0106501), and the National Natural Science Foundation of China (61976073).
abstain abstain abstain objective result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain objective abstain abstain abstain abstain abstain other other other other objective other other abstain method other other abstain other other method other other objective abstain method other other other other abstain other other method other other other other other other other other other abstain other abstain abstain other abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result method other abstain other other other other other other objective objective other other other other other other method objective objective abstain abstain other