Current Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module, where the retriever selects potentially relevant passages from open-source documents for a given question, and the reader produces an answer based on the retrieved passages. The recently proposed Fusion-in-Decoder (FiD) framework is a representative example, which is built on top of a dense passage retriever and a generative reader, achieving the state-of-the-art performance. In this paper we further improve the FiD approach by introducing a knowledge-enhanced version, namely KG-FiD. Our new model uses a knowledge graph to establish the structural relationship among the retrieved passages, and a graph neural network (GNN) to re-rank the passages and select only a top few for further processing. Our experiments on common ODQA benchmark datasets (Natural Questions and TriviaQA) demonstrate that KG-FiD can achieve comparable or better performance in answer prediction than FiD, with less than 40% of the computation cost. Open-Domain Question Answering (ODQA) is the task of answering natural language questions in open domains. A successful ODQA model relies on effective acquisition of world knowledge. A popular line of work treats a large collection of open-domain documents (such as Wikipedia articles) as the knowledge source, and design a ODQA system that consists of a retrieving module and a reading module . The retriever pulls out a small set of potentially relevant passages from the open-source documents for a given question, and the reader produces an answer based on the retrieved passages (Karpukhin et al., 2020; Guu et al., 2020; Izacard and Grave, 2020). An earlier example of this kind is DrQA (Chen et al., 2017), which used  Work done during internship at Microsoft. an traditional search engine based on the bag of words (BoW) document representation with TF-IDF term weighting, and a neural reader for extracting candidate answers for each query based on the dense embedding of the retrieved passages. With the successful development of Pre-trained Language Models (PLMs) in neural network research, dense embedding based passage retrieval (DPR) models (Karpukhin et al., 2020; Qu et al., 2021) have shown superior performance over BoW/TF-IDF based retrieval models due to utilization of contextualized word embedding in DPR, and generative QA readers (Lewis et al., 2020; Roberts et al., 2020) usually outperform extraction based readers (Devlin et al., 2019; Guu et al., 2020) due to the capability of the former in capturing lexical variants with a richer flexibility. The recently proposed Fusion-in-Decoder (FiD) model (Izacard and Grave, 2021) is representative of those methods with a DPR retriever and a generative reader, achieving the state-of-the-art results on ODQA evaluation benchmarks. FiD also significantly improved the scalability of the system over previous generative methods by encoding the retrieved passages independently instead of encoding the concatenation of all retrieved passages (which was typical in previous methods). Inspired by the success of FiD, this paper aims further improvements of the state of the art of ODQA in the paradigm with a DPR retriever and a generative reader. Specifically, we point out two potential weaknesses or limitations of FiD as the rooms for improvements, and we propose a novel solution namely KG-FiD to address these issues with FiD. The two issues are: Issue 1. The independent assumption among passages is not justified. Notice that both the DPR retriever and the generative reader of FiD perform independent encoding of the retrieved passages, which means that they cannot leverage the semantic relationship among passages for passage 4961 embedding and answer generation even if such relational knowledge is available. But we know that rich semantic connections between passages often provide clues for better answering questions (Min et al., 2019). Issue 2. Efficiency Bottleneck . For each input question, the FiD generative reader receives about 100 passages from the DPR module, with a relatively high computational cost. For example, the inference per question takes more than 6 trillion floating-point operations. Simply reducing the number of retrieved passages sent to the reader will not be a good solution as it will significantly decrease the model performance (Izacard and Grave, 2021). How to overcome such inefficient computation issue is a challenging question for the success of FiD in realistic ODQA settings. We propose to address both of the above issues with FiD by leveraging an existing knowledge graph (KG) to establish relational dependencies among retrieved passages, and employing Graph Neural Networks (GNNs) to re-rank and prune retrieved passages for each query. We name our new approach as KG-FiD. Specifically, KG-FiD employs a two-stage passage reranking by applying GNN to model structural and semantic information of passages. Both stages rerank the input passages and only a few top-reranked passages are fed into subsequent modules. The first stage reranks passages returned by the retriever, where we use the passage embeddings generated by DPR as the initial GNN node representation. This allows reranking a much larger set of initial candidate passages to enhance coverage of answers. The second stage performs joint passage reranking and answer generation, where the node embeddings are initialized by the embeddings of passage-question pairs output from the reader encoder. This stage operates on a smaller candidate set but aims for more accurate reranking and passage pruning. To improve the efficiency, in the second-stage reranking, our GNN model adopts representations from the intermediate layer in the reader encoder instead of the final layer to initiate passage node embeddings. Then only a few top reranked passages will be passed into the higher layers of encoder and the decoder for answer generation, while other passages will not be further processed. This is coupled with a joint training of passage reranking and answer generation. As shown in Section 4.3, these strategies significantly reduce the computation cost while still maintaining a good QA performance. Our experiments on ODQA benchmark datasets Natural Questions and TriviaQA demonstrate that KG-FiD can achieve comparable or better performance in answer prediction than FiD, with only 40% of the computation cost of FiD. ODQA with text corpus ODQA usually assumes that a large external knowledge source is accessible and can be leveraged to help answer prediction. For example, previous works (Chen et al., 2017; Karpukhin et al., 2020; Izacard and Grave, 2021) mainly use Wikipedia as knowledge source which contains millions of text passages. In this case, current ODQA models mainly contains a retriever to select related passages and a reader to generate the answer. Thus, the follow-up works mainly aim to: (1) Improve the retriever: from sparse retrieval based on TF-IDF or BM25 (Chen et al., 2017; Yang et al., 2019) to dense retrieval (Karpukhin et al., 2020) based on contextualized embeddings generated by pre-trained language models (PLMs). Moreover, some further improvement are also proposed such as better training strategy (Qu et al., 2021), reranking based on retrieved passages (Wang et al., 2018; Nogueira and Cho, 2019; Mao et al., 2021), and knowledge distillation from reader to retriever (Izacard and Grave, 2020); (2) Improve the reader: changing from Recurrent Neural Network (Chen et al., 2017) to PLMs such as extractive reader BERT (Karpukhin et al., 2020; Iyer et al., 2021; Guu et al., 2020) and generative reader BART and T5 (Izacard and Grave, 2021; Lewis et al., 2020). Besides, some works (Guu et al., 2020; Lewis et al., 2020; Sachan et al., 2021) have shown that additional unsupervised pre-training on retrieval-related language modeling tasks can further improve ODQA performance. However, none of these methods modeled the relationships among different passages. ODQA with knowledge graph Besides the unstructured text corpus, world knowledge also exists in knowledge graphs (KGs), which represent entities and relations in a structural way and have been used in a variety of NLP tasks (Xu et al., 2021b; Yu et al., 2020; Xu et al., 2021a). Some works (Berant et al., 2013; Sun et al., 2018, 2019; Xiong et al., 2019) restrict the answer to be entities in the knowledge graph, while our work focus on more general 4962 ODQA setting where the answer can be any words or phrases. Under this setting, some recent efforts have been made to leverage knowledge graphs for ODQA (Min et al., 2019; Asai et al., 2020; Zhou et al., 2020). For example, UniK-QA (Oguz et al., 2020) transforms KG triplets into text sentences and combine them into text corpus, which loses structure information of KG. Other works use KG to build relationship among passages similar to ours. KAQA (Zhou et al., 2020) use passage graph to propagate passage retrieve scores and answer span scores. Graph-Retriever (Min et al., 2019) iteratively retrieve passages based on the relationship between passages, and also use passage graph to improve passage selection in an extractive reader. However, applying KG to improve the recent advanced FiD framework remains unstudied. In the following sections, we first introduce how to apply KG to build a graph structure among the retrieved passages (Section 3.1). Then we show how we adopt the graph-based stage-1 reranking with DPR retriever to improve passage retrieval (Section 3.2). Next we introduce joint stage-2 reranking and answer generation in the reading module (Section 3.3). Finally we illustrate the improvement of efficiency by using intermediate layer representation for stage-2 reranking (Section 3.4). The overview of our framework is illustrated in Figure 1. 3.1 Construct Passage Graph using KG The intuition behind using KG is that there exists the structural relationship among the retrieved passages which can be captured by the KG. Similar to (Min et al., 2019), we construct the passage graph where vertices are passages of text and the edges represent the relationships that are derived from the external KGs as KG = { ( e h , r, e t ) } , where e h , r, e t are the head entity, relation and tail entity of a triplet respectively. First, we formalize the definition of a passage . Following previous works (Wang et al., 2019; Karpukhin et al., 2020), each article in the text corpus is split into multiple disjoint text blocks of 100 words called passages , which serve as the basic retrieval units. We assume there is a one-one mapping between the KG entities and articles in the text corpus. Specifically, we use English Wikipedia as the text corpus and English Wiki-data (Vrandecic and Krtzsch, 2014) as the knowledge graph, since there exists an alignment between the two resources 1 . For example, for the article titled with New York Yankees, it contains passages such as The New York Yankees are an American professional baseball team .... The article also corresponds to a KG entity with the same name as New York Yankees. Then we define the mapping function e = f ( p ) , where the KG entity e corresponds to the article which p belongs to. Note that one passage can only be mapped to one entity, but multiple passages could be mapped to the same entity. The final passage graph is defined as G = { ( p i , p j ) } , where passages p i and p j are connected if and only if their mapped entities are directly connected in the KG, i.e., ( f ( p i ) , r, f ( p j ))  KG . Since the total number of passages is very large, e.g., more than 20M in Wikipedia, constructing and maintaining a graph over all the passages is inefficient and memory-consuming. Thus, we build a passage graph on the fly for each question, based on the retrieved passages. DPR Retriever: Our framework applies DPR (Karpukhin et al., 2020) as the retriever, which uses a BERT based passage encoder to encode all the N passages in the text corpus { p 1 , p 2 ,    , p N } . Suppose all the passage embeddings are fixed and stored in memory as M  RN  D where D is the hidden dimension: M i = BERT ( p i ) for i  { 1 , 2 ,    N } (1) For an input question q , DPR applies another BERT-based question encoder to obtain its representation Q , then it builds on FAISS (Johnson et al., 2019) to conduct fast dot-product similarity search between Q and M , and returns N 1 ( N 1  N ) passages with the highest similarity scores. Stage-1 Reranking: We see that the DPR retriever returns N 1 passages which are independently retrieved based on the similarity between the question and each passage, without considering inter-passage relationship. Thus instead of directly retrieving N 1 passages for the reader, we propose to first retrieve N 0 ( N 0 > N 1 ) passages, then rerank them and output topN 1 reranked passages into the reader. Following Section 3.1, we construct a graph among the N 0 retrieved passages denoted as G 0 . 1 Entity recognition and linking can be used if there is no such alignment. We aim to rerank the retrieved passages based on both the structural information and the textual semantic information of them. To represent the semantic information of passages, one can use another pre-trained language model to encode the passage texts, but this will not only include lots of additional model parameters, but also incur heavy computational cost as N 0 can be large. To avoid both additional memory and computation cost, we propose to reuse the offline passage embeddings M generated from the DPR retriever in Equation 1 as the initial node representation: E (0) i = M r i where { r i | i  { 1 , 2 ,    , N 0 }} is the set of retrieved passage indices. Then we employ a graph attention network (GAT) (Velickovic et al., 2018) with L g layers as GNN model to update representations for each node based on the passage graph and initial representation. The l -th layer of the GNN model updates the embedding of node i as follows: E ( l ) i = h ( E ( l  1) i , { E ( l  1) j } ( i,j ) G 0 ) (2) where h is usually a non-linear learnable function which aggregates the embeddings of the node itself and its neighbor nodes. The reranking score for each passage p r i is calculated by s stage-1 i = QTE ( L g ) i , where Q is the question embedding also generated by the DPR retriever. Then we sort the retrieved passages by the reranking scores, and input the topN 1 passages into the reader. The training loss of passage ranking for each question is: L stage-1 r =  N 0 (cid:88) i =1 y i log (cid:32) exp( s stage-1 i ) (cid:80) N 0 j =1 exp( s stage-1 j ) (cid:33) (3) where y i = 1 if p r i is the gold passage 2 that contains the answer, and 0 otherwise. As we only add a lightweight graph neural network and reuse the pre-computed and static DPR passage embeddings, our reranking module can process a large number of candidate passages efficiently for each question. In experiments, we set N 0 = 1000 and N 1 = 100 . In this section, we briefly introduce the vanilla FiD reading module before illustrating our joint 2 We follow Karpukhin et al. (2020) on the definition of gold passages. retrieved passages { p a 1 , p a 2 ,    , p a N 1 } as input. Vanilla FiD Reading Module: We denote the hidden dimension as H and number of encoder layers and decoder layers as L , FiD reader first separately encodes each passage p a i concatenated with question q : P (0) i = T5-Embed ( q + p a i )  RT p  H , (4) P ( l ) i = T5-Encoder l ( P ( l  1) i )  RT p  H , (5) where T p is the sequence length of a passage concatenated with the question. T5-Embed (  ) is the initial embedding layer of T5 model (Raffel et al., 2019) and T5-Encoder l (  ) is the l -th layer of its encoder module. Then the token embeddings of all passages output from the last layer of the encoder are concatenated and sent to the decoder to generate the answer tokens A : A = T5-Decoder (cid:16) [ P ( L ) 1 ; P ( L ) 2 ;    ; P ( L ) N 1 ] (cid:17) (6) Stage-2 Reranking: Note that vanilla FiD reader neglect the cross information among passages, and the joint modeling in the decoding process makes it vulnerable to the noisy irrelevant passages. Thus, we propose to leverage the passage graph to rerank the input N 1 passages during the encoding and only select topN 2 ( N 2 < N 1 ) reranked passages into the decoder, which is named as stage-2 reranking. Similar to stage-1 reranking, the reranking model is based on both the structural information and the textual semantic information of passages. We denote the passage graph as G 1 , which is a subgraph of G 0 . To avoid additional computation and memory cost, we propose to reuse the encoder-generated question-aware passage representation from FiD reader for passage reranking as it is already computed in Equation 5. Specifically, the initial node embeddings Z (0) i for passage p a i comes from the first token embedding of the final layer in the FiD-Encoder, i.e., Z (0) i = P ( L ) i (0)  RD . Then same as stage-1 reranking, we also employ a GAT (Velickovic et al., 2018) with L g layers as the graph neural network (GNN) model to update representations for each node based on the passage graph, similar to Equation 2: Z ( L g ) = GAT ( Z (0) , G  1 ) . The reranking score of passage p a i is calculated by s stage-2 i = WTZ ( L g ) i where W is a trainable model parameter. After reranking, only the final topN 2 ( N 2 < N 1 ) passages are sent for decoding. Suppose their indices are { g 1 , g 2 ,    , g N 2 } , the decoding process in Equation 6 becomes: A = T5-Decoder (cid:16) [ P ( L ) g 1 ; P ( L ) g 2 ;    ; P ( L ) g N 2 ] (cid:17) (7) where A is the generated answer. Similar to stage-1 reranking, the training loss of passage ranking for each question is: L stage-2 r =  N 1 (cid:88) i =1 y i log (cid:32) exp( s stage-2 i ) (cid:80) N 1 j =1 exp( s stage-2 j ) (cid:33) (8) where y i = 1 if p a i is the gold passage that contains the answer, and 0 otherwise. The passage reranking and answer generation are jointly trained. We denote the answer generation loss for each question is L a , then the final training loss of our reader module is L = L a +  L stage-2 r , where  is a hyper-parameter which controls the weight of reranking task in the total loss. Note that the first stage reranking is based on DPR embeddings, which are are high-level (one vector per passage) and not further trained. While the second stage is based on reader-generated passage-question embeddings, which are semantic-level and trainable as part of the model output. Thus the second stage can better capture semantic information of passages and aims for more accurate reranking over a smaller candidate set. In the experiment, we set N 1 = 100 and N 2 = 20 . Recall that in the stage-2 reranking, we take the passage representation from the last layer of reader encoder for passage reranking. In this section, we propose to further reduce the computation cost by taking the intermediate layer representation rather than the last layer. The intuition is that answer generation task is more difficult than passage reranking which only needs to predict whether the passage contains the answer or not. Thus we may not need the representation from the whole encoder module for passage reranking. Suppose we take the representation from the L 1 th layer ( 1  L 1 < L ), i.e., Z (0) i = P ( L 1 ) i (0) for i  { 1 , 2 ,    , N 1 } , and the reranking method remains the same. Then only the top N 2 ( N 2 < N 1 ) reranked passages will go through the rest layers of FiD-encoder. Suppose their indices are 4965 I g = { g 1 , g 2 ,    , g N 2 } , for l  L 1 + 1 : P ( l ) i = (cid:40) T5-Encoder l ( P ( l  1) i ) if i  I g Stop-Computing else (9) Then P ( L ) g 1 , P ( L ) g 2 ,    , P ( L ) g N 2 are sent into the decoder for answer generation as in Equation 7. In Section 4.3, we demonstrate this can reduce 60% computation cost than the original FiD while keeping the on-par performance on two benchmark datasets. Here we analyze the theoretical time complexity of our proposed KG-FiD compared to vanilla FiD. More practical computation cost comparison is shown in Appendix A.5. Because both the computations of DPR retrieving and stage-1 reranking are negligible compared to the reading part, we only analyze the reading module here. Suppose the length of answer sequence A is denoted as T a and the average length of the passage (concatenated with question) is T p . For vanilla FiD reader, the time complexity of the encoder module is O ( L  N 1  T 2 p ) , where L, N 1 denote the number of encoder layers and the number of passages for reading. The square comes from the self-attention mechanism. The decoder time complexity is O ( L  ( N 1  T p  T a + T 2 a )) , where N 1  T p  T a comes from the cross-attention mechanism. For our reading module, all the N 1 candidate passages are processed by the first L 1 layers of encoder. But only N 2 passages are processed by the remaining L  L 1 encoder layers and sent into the decoder. Thus, the encoder computation complexity becomes O (( L 1  N 1 +( L  L 1 )  N 2 )  T 2 p ) , and the decoder computation takes O ( L  ( N 2  T p  T a + T 2 a )) . Because L 1 < L, N 2 < N 1 , both the encoding and decoding of our method is more efficient than vanilla FiD. Furthermore, the answer is usually much shorter than the passage (which is the case in our experi-ments), i.e., T a  T p . Then the decoding computation can be negligible compared to the encoding. In this case, the approximated ratio of saved computation cost brought by our proposed method is: S = 1  ( L 1  N 1 + ( L  L 1 )  N 2 )  T 2 p L  N 1  T 2 p = (1  L 1 L )(1  N 2 N 1 ) This shows that we can reduce more computation cost by decreasing L 1 or N 2 . For example, if setting L 1 = L/ 4 , N 2 = N 1 / 5 , we can reduce about 60% of computation cost. More empirical results and discussions will be presented in Section 4.3. In this section, we conduct extensive experiments on two most commonly-used ODQA benchmark datasets: Natural Questions (NQ) (Kwiatkowski et al., 2019) which is based on Google Search Queries, and TriviaQA (Joshi et al., 2017) which contains questions from trivia and quiz-league web-sites. We follow the same setting as (Izacard and Grave, 2021) to preprocess these datasets, which is introduced in Appendix A.1. All our experiments are conducted on 8 Tesla A100 40GB GPUs. Knowledge Source: Following (Karpukhin et al., 2020; Izacard and Grave, 2021), we use the English Wikipedia as the text corpus, and apply the same preprocessing to divide them into disjoint passages with 100 words, which produces 21M passages in total. For the knowledge graph, we use English Wikidata. The number of aligned entities, relations and triplets among these entities are 2.7M, 974 and 14M respectively. Model Details: For the retrieving module, we use the DPR retriever (Karpukhin et al., 2020) which contains two BERT (base) models for encoding question and passage separately. For the GNN reranking models, we adopt 3-layer Graph Attention Networks (GAT) (Velickovic et al., 2018). For the reading module, same as (Izacard and Grave, 2021), we initialize it with the pretrained T5-base and T5-large models (Raffel et al., 2019), and we name the former one as KG-FiD (base) and the latter one as KG-FiD (large). Our implementation is based on the HuggingFace Transformers library (Wolf et al., 2019). For number of passages, we set N 0 = 1000 , N 1 = 100 , N 2 = 20 . The training process of our method is introduced in Appendix A.3. More results about model design and hyper-parameter search is in Appendix A.4. Evaluation: We follow the standard evaluation metric of answer prediction in ODQA, which is the exact match score (EM) (Rajpurkar et al., 2016). A generated answer is considered correct if it matches any answer in the list of acceptable answers after 4966 normalization 3 . For all the experiments, we conduct 5 runs with different random seeds and report the averaged scores. We mainly compare KG-FiD with the baseline model FiD (Izacard and Grave, 2021). For other baselines, we compare with representative methods from each category: (1) not using external knowledge source: T5 (Roberts et al., 2020) and GPT-3 (Brown et al., 2020); (2) reranking-based methods: RIDER (Mao et al., 2021) and RECONSIDER (Iyer et al., 2021); (3) leveraging knowledge graphs or graph information between passages: Graph-Retriever (Min et al., 2019), Path-Retriever (Asai et al., 2020), KAQA (Zhou et al., 2020), and UniK-QA (Oguz et al., 2020). We also compare with methods (4) with additional large-scale pre-training: REALM (Guu et al., 2020), RAG (Lewis et al., 2020) and Joint Top-K (Sachan et al., 2021). Comparison with Baselines: Table 1 shows the results of our method and all baselines. We see that our proposed model KG-FiD consistently and significantly improves FiD on both NQ and TriviaQA datasets over both base and large model. Specifically, for large model, KG-FiD improves FiD by 1 . 5% and 1 . 1% on two datasets respectively, which has larger improvement compared to base model. We think the reason is that more expressive reader will also benefit the stage-2 reranking since the initial passage embeddings are generated by the reader encoder module. We also see that our proposed method outperforms all the baseline methods except UniK-QA (Oguz et al., 2020). However, UniK-QA uses additional knowledge source Wikipedia-Table for retrieval, which is highly related with the NQ dataset and makes it unfair to directly compare with our method. Efficiency & Accuracy: Table 2 show the detailed comparison between our method and FiD in the large model version. The results of base model version is shown in Appendix A.4. Besides EM score, we also report the ratio of computation flops (#FLOPs) and inference latency (per question). The detailed calculation of #FLOPs is shown in Appendix A.5. From table 2, we see 3 The normalization includes lowercasing and removing articles, punctuation and duplicated whitespace. that (1) for KG-FiD, decreasing L 1 can improve the computation efficiency as analyzed in Section 3.4, while increasing L 1 can improve the model performance. We think the performance improvement comes from the noise reduction of passage filtering. For a larger L 1 , the passage embeddings for reranking will have a better quality so that the gold passages are less likely to be filtered out. (2) Simply reducing the number of passages N 1 into vanilla FiD reader can reduce computation cost, but the performance will also drop significantly (from 51.9 to 50.3 on NQ dataset). (3) Our model can achieve the performance on par with FiD with only 38% of computation cost. When consuming the same amount of computations ( L 1 = 24 ), our model significantly outperforms FiD on both NQ and TriviaQA datasets. These experiments demonstrate that our model is very flexible and can improve both the efficiency and effectiveness by changing L 1 . Effect of Each Reranking Stage: Since our proposed graph-based reranking method are applied in both retrieving stage (Section 3.2) and reading stage (Section 3.3). We conduct ablation study 4967 Model #FLOPs NQ TriviaQA EM Latency (s) EM Latency (s) FiD (N 1 =40) 0.40x 50.3 0.74 (0.45x) 67.5 0.73 (0.44x) FiD (N 1 =100) 1.00x 51.9 1.65 (1.00x) 68.7 1.66 (1.00x) KG-FiD (N 1 =100, L 1 =6) 0.38x 52.0 0.70 (0.42x) 68.9 0.68 (0.41x) KG-FiD (N 1 =100, L 1 =12) 0.55x 52.3 0.96 (0.58x) 69.2 0.94 (0.57x) KG-FiD (N 1 =100, L 1 =18) 0.72x 52.6 1.22 (0.74x) 69.8 1.22 (0.73x) KG-FiD (N 1 =100, L 1 =24) 0.90x 53.4 1.49 (0.90x) 69.8 1.48 (0.89x) Table 2: Inference #FLOPs, Latency (second) and Exact match score of FiD (large) and KG-FiD (large). to validate the effectiveness of each one. Table 3 shows the experiment results by removing each module. We see the performance of KG-FiD drops when removing any of the two reranking modules, demonstrating both of them can improve model performance. Another thing we observe is that stage-1 reranking is more effective in base model while stage-2 reranking is more effective in large model. This is reasonable since stage-2 reranking relies on the effectiveness of reader encoder module, where the large model is usually better than the base model. Passage Ranking Results: We additionally show that our proposed GNN reranking method can improve the passage retrieval results. This is demonstrated in Figure 2, where we report Hits@K metric over NQ test set, measuring the percentage of top-K retrieved passages that contain the gold passages (passages that contain the answer). We see that DPR+stage-1 reranking consistently outperforms DPR for all the K  { 10 , 20 , 50 , 100 } . With two stages of reranking, the retrieval results are further improved for K  { 10 , 20 } (We only cares about K  20 for stage-2 reranking since N 2 = 20 ). This shows that such reranking can increase the rank of gold passages which are previ-10 20 50 100 K 76 78 80 82 84 86 88 H i t s @ KDPR DPR+stage-1 DPR+stage-1&2 Figure 2: Passage ranking results over NQ test set of DPR retriever and our proposed two-stage rerankings over base model. ously ranked lower by DPR retriever and improve the efficacy of passage pruning. This work tackles the task of Open-Domain Question Answering. We focus on the current best performed framework FiD and propose a novel KG-based reranking method to enhance the cross-modeling between passages and improve computation efficiency. Our two-stage reranking methods reuses the passage representation generated by DPR retriver and the reader encoder and apply graph neural networks to compute reranking scores. We further propose to use the intermediate layer of encoder to reduce computation cost while still maintaining good performance. Experiments on Natural Questions and TriviaQA show that our model can significantly improve original FiD by 1 . 5% exact match score and achieve on-par performance with FiD but reducing over 60% of computation cost. We thank all the reviewers for their valuable comments. We also thank Woojeong Jin, Dong-Ho Lee, and Aaron Chan for useful discussions. Donghan Yu and Yiming Yang are supported in part by the United States Department of Energy via the Brookhaven National Laboratory under Contract No. 384608.
abstain abstain result objective objective abstain abstain abstain abstain abstain abstain other abstain abstain objective objective abstain abstain abstain method abstain abstain abstain abstain abstain abstain objective objective abstain abstain objective abstain abstain abstain result abstain abstain abstain objective other other other other other other other other method other other other other other other method method method method method abstain method method abstain method method abstain abstain method abstain abstain abstain method method other method method method abstain method abstain method method abstain abstain method abstain method method method method abstain method abstain other method abstain method method abstain method abstain abstain abstain other abstain method abstain abstain abstain method method method abstain method method abstain other method method abstain method abstain abstain abstain abstain method abstain abstain method method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain result result abstain abstain other abstain abstain abstain result abstain abstain abstain result result abstain abstain method objective method objective result abstain other other other