One of the main challenges in conversational question answering (CQA) is to resolve the conversational dependency, such as anaphora and ellipsis. However, existing approaches do not explicitly train QA models on how to resolve the dependency, and thus these models are limited in understanding human dialogues. In this paper, we propose a novel framework, EXCORD ( Ex plicit guidance on how to resolve Co nve r sational D ependency) to enhance the abilities of QA models in comprehending conversational context. EXCORD first generates self-contained questions that can be understood without the conversation history, then trains a QA model with the pairs of original and self-contained questions using a consistency-based regularizer. In our experiments, we demonstrate that EXCORD significantly improves the QA models' performance by up to 1.2 F1 on QuAC (Choi et al., 2018), and 5.2 F1 on CANARD (Elgohary et al., 2019), while addressing the limitations of the existing approaches. 1 1 Introduction Conversational question answering (CQA) involves modeling the information-seeking process of humans in a dialogue. Unlike single-turn question answering (QA) tasks (Rajpurkar et al., 2016; Kwiatkowski et al., 2019), CQA is a multi-turn QA task, where questions in a dialogue are context-dependent; 2 hence they need to be understood with the conversation history (Choi et al., 2018; Reddy et al., 2019). As illustrated in Figure 1, to answer  Corresponding author 1 Our models and code are available at: https://github.com/dmis-lab/excord 2 While the term context usually refers to the evidence document from which the answer is extracted, in CQA, it refers to conversational context. the current question  Was he close with anyone else?,  a model should resolve the conversational dependency, such as anaphora and ellipsis, based on the conversation history. A line of research in CQA proposes the end-to-end approach, where a single QA model jointly encodes the evidence document, the current question, and the whole conversation history (Huang et al., 2018; Yeh and Chen, 2019; Qu et al., 2019a). In this approach, models are required to automatically learn to resolve conversational dependencies. However, existing models have limitations to do so without explicit guidance on how to resolve these dependencies. In the example presented in Figure 1, models are trained without explicit signals that  he  refers to  Leonardo da Vinci,  and  anyone else  can be more elaborated with  other than his pupils, Salai and Melzi . Another line of research proposes a pipeline approach that decomposes the CQA task into question rewriting (QR) and QA, to reduce the complexity of the task (Vakulenko et al., 2020). Based on the conversation history, QR models first generate self-contained questions by rewriting the original questions, such that the self-contained questions can be understood without the conversation history. For instance, the current question q 3 is reformulated as the self-contained question  q 3 by a QR model in Figure 1. After rewriting the question, QA models are asked to answer the self-contained questions rather than the original questions. In this approach, QA models are trained to answer relatively simple questions whose dependencies have been resolved by QR models. Thus, this limits reasoning abilities of QA models for the CQA task, and causes QA models to rely on QR models. In this paper, we emphasize that QA models can be enhanced by using both types of questions with explicit guidance on how to resolve the conversational dependency. Accordingly, we propose EXCORD ( Ex plicit guidance on how to Resolve Co nve r sational D ependency), a novel training framework for the CQA task. In this framework, we first generate self-contained questions using QR models. We then pair the self-contained questions with the original questions, and jointly encode them to train QA models with consistency regularization (Laine and Aila, 2016; Xie et al., 2019). Specifi-cally, when original questions are given, we encourage QA models to yield similar answers to those when self-contained questions are given. This training strategy helps QA models to better understand the conversational context, while circumventing the limitations of previous approaches. To demonstrate the effectiveness of EXCORD, we conduct extensive experiments on the three CQA benchmarks. In the experiments, our framework significantly outperforms the existing approaches by up to 1.2 F1 on QuAC (Choi et al., 2018) and by 5.2 F1 on CANARD (Elgohary et al., 2019). In addition, we find that our framework is also effective on a dataset CoQA (Reddy et al., 2019) that does not have the self-contained questions generated by human annotators. This indicates that the proposed framework can be adopted on various CQA datasets in future work. We summarize the contributions of this work as follows:  We identify the limitations of previous approaches and propose a unified framework to address these. Our novel framework improves QA models by incorporating QR models, while reducing the reliance on them. Our framework encourages QA models to learn how to resolve the conversational dependency via consistency regularization. To the best of our knowledge, our work is the first to apply the consistency training framework to the CQA task. We demonstrate the effectiveness of our framework on three CQA benchmarks. Our framework is model-agnostic and systematically improves the performance of QA models. In CQA, a single instance is a dialogue, which consists of an evidence document d , a list of questions q = [ q 1 , ..., q T ] , and a list of answers for the questions a = [ a 1 , ..., a T ] , where T represents the number of turns in the dialogue. For the t -th turn, the question q t and the conversation history H t = [( q 1 , a 1 ) , ..., ( q t  1 , a t  1 )] are given, and a model should extract the answer from the evidence document as:  a t = arg max a t P( a t | d, q t , H t ) (1) where P(  ) represents a likelihood function over all the spans in the evidence document, and  a t is the predicted answer. Unlike single-turn QA, since the current question q t is dependent on the conversation history H t , it is important to effectively encode the conversation history and resolve the conversational dependency in CQA. A naive approach in solving CQA is to train a model in an end-to-end manner (Figure 2a). Since standard QA models generally are ineffective in the CQA task, most studies attempt to develop a QA model structure or mechanism for encoding the conversation history effectively (Huang et al., 2018; Yeh and Chen, 2019; Qu et al., 2019a,b). Although Answer QA Loss Evidence Document QuestionAnswering Current Question Conversation History these efforts improved performance on the CQA benchmarks, existing models remain limited in understanding conversational context. In this paper, we emphasize that QA models can be further improved with explicit guidance using self-contained questions effectively. Recent studies decompose the task into two subtasks to reduce the complexity of the CQA task. The first sub-task, question rewriting, involves generating self-contained questions by reformulating the original questions. Neural-net-based QR models are commonly used to obtain self-contained questions (Lin et al., 2020; Vakulenko et al., 2020). The QR models are trained on the CANARD dataset (Elgohary et al., 2019), which consists of 40K pairs of original QuAC questions and their self-contained versions that are generated by human annotators. After generating the self-contained questions, the next sub-task, question answering, is carried out. Since it is assumed that the dependencies in the questions have already been resolved by QR models, existing works usually use standard QA models (not specialized to CQA); however conversational QA models can also be used (the dotted line in Figure 2b). We formulate the process of predicting the answer in the pipeline approach as: P( a t | d, q t , H t )  P rewr ( q t | q t , H t )  P read ( a t | d,  q t ) (2) where P rewr (  ) and P read (  ) are the likelihood functions of QR and QA models, respectively. q t is a self-contained question rewritten by the QR model. The main limitation of the pipeline approach is that QA models are never trained on the original questions, which limits their abilities to understand the conversational context. Moreover, this approach makes QA models dependent on QR models; hence QA models suffer from the error propagation from QR models. 3 On the other hand, our framework enhances QA models' reasoning abilities for CQA by jointly utilizing original and self-contained questions. In addition, QA models in our framework do not rely on QR models at inference time and thus do not suffer from error propagation. We introduce a unified framework that jointly encodes the original and self-contained questions as illustrated in Figure 2c. Our framework consists of two stages: (1) generating self-contained questions using a QR model (3.1) and (2) training a QA model with the original and self-contained questions via consistency regularization (3.2). Similar to the pipeline approach, we utilize a QR model to obtain self-contained questions. We use the obtained questions for explicit guidance in the next stage. As shown in Equation 2, the QR task is to generate a self-contained question given an original question and a conversation history. Following Lin et al. (2020), we adopt a T5-based sequence generator (Raffel et al., 2020) as our QR model, which achieves comparable performance with that of humans in QR. 4 For training and evaluating the QR model, we use the CANARD dataset following previous works on QR (Lin et al., 2020; Vakulenko et al., 2020). During inference, we utilize the top-k random sampling decoding based on beam search with the adjustment of the softmax temperature (Fan et al., 2018; Xie et al., 2019). Our goal is to enhance the QA model's ability to understand conversational context. Accordingly, we use consistency regularization (Laine and Aila, 2016; Xie et al., 2019), which enforces a model to make consistent predictions in response to transformations to the inputs. We encourage the model's predicted answers from the original questions to be similar to those from the self-contained questions (3.1). Our consistency loss is defined as: L cons t = KL (P read  ( a t | d, q t , H t ) || P read   ( a t | d,  q t ,  H t )) (3) where KL (  ) represents the KullbackLeibler divergence function between two probability distributions. is the model's parameters, and   depicts a fixed copy of  . With the consistency loss, QA models are regularized to make consistent predictions, regardless of whether the given question is self-contained or not. In order to output an answer distribution that is closer to P read   ( a t | d,  q t ,  H t ) , QA models should treat original questions as if they were rewritten into self-contained questions by referring to the 4 On CANARD, our QR model achieved comparable performance with the human performance in preliminary experiments. conversation history. Through this process, our consistency regularization method serves as explicit guidance that encourages QA models to resolve the conversational dependency. In our framework, P read  ( a t | ) is the answer span distribution over all evidence document tokens. In contrast to Asai and Hajishirzi (2020), by using all probability values in the answer distributions, the signals of self-contained questions can be effectively propagated to the QA model. In addition to using all probability values, we also sharpened the target distribution P read   ( a t | d,  q t ,  H t ) by adjusting the temperature (Xie et al., 2019) to strengthen the QA model's training signal. Finally, we calculate the final loss as: L = L orig +  1 L self +  2 L cons (4) where  1 and  2 are hyperparameters. L orig and L self are calculated by the negative log-likelihood between the predicted answers and gold standards given the original and self-contained questions, respectively. Comparison with previous works Consistency training has mainly been studied as a method for regularizing model predictions to be invariant to small noises that are injected into the input samples (Sajjadi et al., 2016; Laine and Aila, 2016; Miyato et al., 2016; Xie et al., 2019). The intuition behind consistency training is to push noisy inputs closer towards their original versions. Therefore, only the original parameters (i.e.,  ) are updated, while the copied model parameters (i.e.,   ) are fixed. In contrast to the original concept of consistency training, our goal is to go in the opposite direction and update the original parameters. Thus, we fix the parameters   with self-contained questions, and soley update  for each training step as shown in Equation 3. QuAC QuAC (Choi et al., 2018) comprises 100k QA pairs in information-seeking dialogues, where a student asks questions based on a topic with background information provided, and a teacher provides the answers in the form of text spans in Wikipedia documents. Since the test set is only available in the QuAC challenge, we evaluate models on the development set. 5 For validation, we use a subset of the original training set of QuAC, which consists of questions that correspond to the self-contained questions in CANARD's development set. The remaining data is used for training. CANARD CANARD (Elgohary et al., 2019) consists of 31K, 3K, and 5K QA pairs for training, development, and test sets, respectively. The questions in CANARD are generated by rewriting a subset of the original questions in QuAC. We use the training and development sets for training and validating QR models, and the test set for evaluating QA models. CoQA CoQA (Reddy et al., 2019) consists of 127K QA pairs and evidence documents in seven domains. In terms of the question distribution, CoQA significantly differs from QuAC (see 5.3). We use CoQA to test the transferability of EXCORD, where a QR model trained on CANARD generates the self-contained questions in a zero-shot manner. Subsequently, we train a QA model by using the original and synthetic questions. Similar to QuAC, the test set of CoQA is soley available in the CoQA challenge. 6 Therefore, we randomly sample 5% of the QA dialogues in the training set and adopt them as our development set. Following Choi et al. (2018), we use the F1, HEQ-Q, and HEQ-D for QuAC and CANARD. HEQ-Q measures whether a model finds more accurate answers than humans (or the same answers) in a given question. HEQ-D measures the same thing, but in a given dialog instead of a question. For CoQA, we report the F1 scores for each domain (children's story, literature from Project Gutenberg, middle and high school English exams, news articles from CNN, Wikipedia) and the overall F1 score, as suggested by Reddy et al. (2019). Note that the baseline approaches and our framework do not limit the structure of QA models. For a fair comparison of the baseline approaches and EXCORD, we test the same QA models in all approaches. The selected QA models are commonly used and have been proven to be effective in CQA. BERT BERT (Devlin et al., 2019) is a contextualized word representation model that is pretrained on large corpora. BERT also works well on CQA datasets, although it is not designed for CQA. It receives the evidence document, current question, and conversation history of the previous turn as input. BERT+HAE BERT+HAE is a BERT-based QA model with a CQA-specific module. Following Qu et al. (2019a), we add the history answer embedding (HAE) to BERT's word embeddings. HAE encodes the information of the answer spans from the previous questions. RoBERTa RoBERTa (Liu et al., 2019) improves BERT by using pretraining techniques to obtain the robustly optimized weights on larger corpora. In our experiments, we found that RoBERTa performs well in CQA, achieving comparable performance with the previous SOTA model, HAM (Qu et al., 2019b), on QuAC. Thus, we adopt RoBERTa as our main baseline model owing to its simplicity and effectiveness. It receives the same input as BERT, otherwise specified. The CANARD training set provides 31,527 self-contained questions from the original QuAC questions. Therefore, we can obtain 31,527 pairs of original and self-contained questions without question rewriting. For the rest of the original questions, we automatically generate self-contained questions by using our QR model. Finally, we obtain 83,568 question pairs and use them in our consistency training. We denote the original questions, self-contained questions generated by humans, and self-contained questions generated by a QR model as Q ,  Q human , and  Q syn , respectively. Additional implementation details are described in Appendix B 4.5 Results Table 1 presents the performance comparison of the baseline approaches to our framework on QuAC and CANARD. Compared to the end-to-end approach, EXCORD consistently improves the performance of QA models on both datasets. Also, these improvements are significant: EXCORD improves the performance of the RoBERTa by absolutely 1.2 and 2.3 F1 scores and BERT by 1.2 and 5.2 F1 scores on QuAC and CANARD, respectively. From these results, we conclude that the consistency training with original and self-contained QA Model Approach QuAC CANARD F1 HEQ-Q HEQ-D F1 HEQ-Q HEQ-D BERT End-to-end 61.5 57.1 5.0 57.4 52.9 3.2 Pipeline 61.2 (0.3) 56.8 (0.3) 5.0 () 62.2 (+ 4.8) 57.8 (+ 4.9) 6.0 (+ 2.8) Ours 62.7 (+ 1.2) 58.4 (+ 1.3) 6.0 (+ 1.0) 62.6 (+ 5.2) 58.2 (+ 5.3) 6.4 (+ 3.2) BERT+HAE End-to-end 62.0 57.3 5.5 58.2 53.5 5.5 Pipeline 61.1 (0.9) 56.3 (1.0) 5.0 (0.5) 62.4 (+ 4.2) 57.8 (+ 4.3) 6.0 (+ 0.5) Ours 63.2 (+ 1.2) 58.9 (+ 1.6) 5.7 (+ 0.2) 63.1 (+ 4.9) 58.4 (+ 4.9) 5.7 (+ 0.2) RoBERTa End-to-end 66.5 62.4 7.2 65.8 62.2 7.1 Pipeline 65.2 (1.3) 60.9 (1.5) 7.1 (0.1) 66.9 (+ 1.1) 63.2 (+ 1.0) 7.3 (+ 0.2) Ours 67.7 (+ 1.2) 64.0 (+ 1.6) 9.3 (+ 2.1) 68.1 (+ 2.3) 64.2 (+ 2.0) 8.4 (+ 1.3) Table 1: Comparison in performance of the baseline approaches and our framework on QuAC and CANARD. On QuAC, the pipeline approach underperforms the end-to-end approach in all baseline models. This indicates that training a QA model soley with self-contained questions is ineffective when human rewrites are not given at the inference phase. On the other hand, EXCORD improves QA models by using both types of questions. As presented in Table 1, our framework significantly outperforms the baseline approaches on QuAC. On CANARD, the pipeline approach is significantly more effective than the end-to-end approach. Since QA models are trained with self-contained questions in the pipeline approach, they perform well on CANARD questions. Nevertheless, EXCORD still outperforms the pipeline approach in most cases. Compared to the pipeline approach, our framework improves the performance of RoBERTa by absolutely 1.2 F1 score. We elaborate on analyses regarding component ablation and transferability. We also describe a case study carried out to highlight such differences between our and baseline approaches. In this section, we comprehensively explore the factors contributing to this improvement in detail: (1) using self-contained questions that are rewritten by humans (  Q human ) as additional data, (2) using self-contained questions that are synthetically generated by the QR model (  Q syn ), and (3) training a QA model with our consistency framework. In Table 2, we present the performance gaps when each component is removed from our framework. We use RoBERTa on QuAC in this experiment. We first explore the effects of  Q human and  Q syn . As shown in Table 2, excluding  Q human degrades the performance of RoBERTa in our framework. Although automatically generated,  Q syn contributes to the performance improvement. Therefore, both types of self-contained questions are useful in our framework. To investigate the effect of our framework, we simply augment  Q human and  Q syn to Q orig , which is called Question Augment (question data augmen-tation). We find that Question Augment slightly improves the performance of RoBERTa on CANARD, whereas it degrades the performance on QuAC. This shows that simply augmenting the questions is ineffective and does not guarantee improvement. On the other hand, our consistency training approach significantly improves performance, showing that EXCORD is a more optimal way to utilizing self-contained questions. We analyze several cases that the baseline approaches answered incorrectly, but our framework answered correctly. We also explore how our framework improves the reasoning ability of QA models, compared to the baseline approaches. These cases Error case # 1 Title : Montgomery Clift Section Title : Film career Document d :    His second movie was The Search . Clift was unhappy with the quality of the script, and edited it himself. The movie was awarded a screenwriting Academy Award for the credited writers. The first case in Table 3 shows the predictions of the two RoBERTa models trained in the end-to-end approach and our framework, respectively. Note that the film in the current question does not refer to The Search (red box) in the document d , but Red River (blue box) in a 1 . When trained in the end-to-end approach, the model failed to comprehend the conversational context and misunderstood what the film refers to, resulting in an incorrect prediction. On the other hand, when trained in EXCORD, the model predicted the correct answer because it enhances the ability to resolve conversational dependency. In the second case, we compare the pipeline approach to EXCORD. In this case, the QR model misunderstood my in the current question as a pronoun and replaced it with the band's name, Train's. Consequently, the QA model received the erroneous self-contained question, resulting in an incorrect prediction. On the other hand, the QA model trained in our framework predicted the correct answer based on the original question q 6 . We train a QR model to rewrite QuAC questions into CANARD questions. Then, self-contained questions can be generated for the samples that do not have human rewrites. This results in the improvement of QA models' performance on QuAC and CANARD (4.5). However, it is questionable whether the QR model can successfully rewrite questions when the original questions significantly differ from those in QuAC. To answer this, we test our framework on another CQA dataset, CoQA. We first analyze how the question distributions of QuAC and CoQA differ. We found that question types in QuAC and CoQA are significantly different, such that QR models could suffer from the gap of question distributions between two datasets. (See details in Appendix A). To test the transferability of EXCORD, we compare the end-to-end approach to our framework on the CoQA dataset. Using a QR model trained on QA model CoQA CANARD, we generate the self-contained questions for CoQA and train QA models with our framework. As presented in Table 4, our framework performs well on CoQA. The improvement in BERT is 0.5 based on the overall F1, and the performance of RoBERTa is also improved by an overall F1 of 0.6. Improvements are also consistent in most of the documents' domains. Therefore, we conclude that our framework can be simply extended to other datasets and improve QA performance even when question distributions are significantly different. We plan to improve the transferability of our framework by fine-tuning QR models on target datasets in future work. Conversational Question Answering Recently, several works introduced CQA datasets such as QUAC (Choi et al., 2018) and COQA (Reddy et al., 2019). We classified proposed methods to solve the datasets into two approaches: (1) end-to-end and (2) pipeline. Most works based on the end-to-end approach focused on developing a model structure (Zhu et al., 2018; Ohsugi et al., 2019; Qu et al., 2019a,b) or training strategy such as multitask with rationale tagging (Ju et al., 2019) that are specialized in the CQA task or datasets. Several works demonstrated the effectiveness of the flow mechanism in CQA (Huang et al., 2018; Chen et al., 2019; Yeh and Chen, 2019). With the advent of a dataset consisting of self-contained questions rewritten by human annotators (Elgohary et al., 2019), the pipeline approach has drawn attention as a promising method for CQA in recent days (Vakulenko et al., 2020). The approach is particularly useful for the open-domain CQA or passage-retrieval (PR) tasks (Dalton et al., 2019; Ren et al., 2020; Anantha et al., 2020; Qu et al., 2020) since self-contained questions can be fed into existing non-conversational search engines such as BM25. Note that our framework can be used jointly with the pipeline approach in the open-domain setting because our framework can improve QA models' ability to find the answers from the retrieved documents. We will test our framework in the open-domain setting in future work. Question Rewriting QR has been studied for augmenting training data (Buck et al., 2018; Sun et al., 2018; Zhu et al., 2019; Liu et al., 2020) or clarifying ambiguous questions (Min et al., 2020). In CQA, QR can be viewed as a task of simplifying difficult questions that include anaphora and ellipsis in a conversation. Elgohary et al. (2019) first proposed the question rewriting task as a sub-task of CQA and the CANARD dataset for the task, which consists of pairs of original and self-contained questions that are generated by human annotators. Vakulenko et al. (2020) used a coreference-based model (Lee et al., 2018) and GPT-2 (Radford et al., 2019) as QR models and tested the models in the QR and PR tasks. Lin et al. (2020) conducted the QR task using T5 (Raffel et al., 2020) and achieved on performance comparable to humans on CANARD. Following Lin et al. (2020), we use T5 in our experiments to generate high-quality questions for enhancing QA models. Consistency Training Consistency regularization (Laine and Aila, 2016; Sajjadi et al., 2016) has been mainly explored in the context of semi-supervised learning (SSL) (Chapelle et al., 2009; Oliver et al., 2018), which has been adopted in the textual domain as well (Miyato et al., 2016; Clark et al., 2018; Xie et al., 2020). However, the consistency training framework is also applicable when only the labeled samples are available (Miyato et al., 2018; Jiang et al., 2019; Asai and Hajishirzi, 2020). The consistency regularization requires adding noise to the sample, which can be either discrete (Xie et al., 2020; Asai and Hajishirzi, 2020) or continuous (Miyato et al., 2016; Jiang et al., 2019). Existing works regularize the predictions of the perturbed samples to be equivalent to be that of the originals'. On the other hand, our method encourages the models' predictions for the original asnwers to be similar to those from the rewritten questions, i.e., synthetic ones. We propose a consistency training framework for conversational question answering, which enhances QA models' abilities to understand conversational context. Our framework leverages both the original and self-contained questions for explicit guidance on how to resolve conversational dependency. In our experiments, we demonstrate that our framework significantly improves the QA model's performance on QuAC and CANARD, compared to the existing approaches. In addition, we veri-fied that our framework can be extended to CoQA. In future work, the transferability of our framework can be further improved by fine-tuning the QR model on target datasets. Furthermore, future work would include applying our framework to the open-domain setting. We thank Sean S. Yi, Miyoung Ko, and Jinhyuk Lee for providing valuable comments and feedback. This research was supported by the MSIT (Min-istry of Science and ICT), Korea, under the ICT Creative Consilience program (IITP-2021-2020-0-01819) supervised by the IITP (Institute for Information communications Technology Planning Evaluation). This research was also supported by National Research Foundation of Korea (NRF-2020R1A2C3010638).
abstain abstain objective abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective objective method method abstain objective result result abstain objective objective method objective objective result other other other other other other other objective other other other other other other method other other other method method method other method abstain abstain abstain abstain method method abstain method method method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain result abstain result abstain abstain result abstain abstain abstain result abstain abstain abstain result abstain result abstain method method abstain result abstain abstain abstain result abstain result method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain result abstain method abstain method method abstain abstain abstain result other other other other objective other other other other other other other other other other other other other other other other other other other abstain abstain other other other other other other other other other other other other other other other other other other other other other method other other other other other other other other other other other other other method objective abstain objective objective result method other other other other other other other other