Abstractive summarization for long-document or multi-document remains challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing long-distance relations in text. In this paper, we present BASS, a novel framework for Boosting Abstractive Summarization based on a unified Semantic graph, which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases. Further, a graph-based encoder-decoder model is proposed to improve both the document representation and summary generation process by leveraging the graph structure. Specifically, several graph augmentation methods are designed to encode both the explicit and implicit relations in the text while the graph-propagation attention mechanism is developed in the decoder to select salient content into the summary. Empirical results show that the proposed architecture brings substantial improvements for both long-document and multi-document summarization tasks. Nowadays, the sequence-to-sequence (Seq2Seq) based summarization models have gained unprecedented popularity (Rush et al., 2015; See et al., 2017; Lewis et al., 2020). However, complex summarization scenarios such as long-document or multi-document summarization (MDS), still bring great challenges to Seq2Seq models (Cohan et al., 2018; Liu et al., 2018). In a long document numerous details and salient content may distribute evenly (Sharma et al., 2019) while multiple documents may contain repeated, redundant or contradictory information (Radev, 2000). These problems make Seq2Seq models struggle with content selection and organization which mainly depend  Work is done during an internship at Baidu Inc. on the long source sequence (Shao et al., 2017). Thus, how to exploit deep semantic structure in the complex text input is a key to further promote summarization performance. Compared with sequence, graph can aggregate relevant disjoint context by uniformly representing them as nodes and their relations as edges. This greatly benefits global structure learning and long-distance relation modeling. Several previous works have attempted to leverage sentence-relation graph to improve long sequence summarization, where nodes are sentences and edges are similarity or discourse relations between sentences (Li et al., 2020). However, the sentence-relation graph is not flexi-ble for fine-grained (such as entities) information aggregation and relation modeling. Some other works also proposed to construct local knowledge graph by OpenIE to improve Seq2Seq models (Fan et al., 2019; Huang et al., 2020). However, the OpenIE-based graph only contains sparse relations between partially extracted phrases, which cannot reflect the global structure and rich relations of the overall sequence. For better modeling the long-distance relations and global structure of a long sequence, we propose to apply a phrase-level unified semantic graph to facilitate content selection and organization. Based on fine-grained phrases extracted from dependency parsing, our graph is suitable for information aggregation with the help of coreference resolution that substantially compresses the input and benefits content selection. Furthermore, relations between phrases play an important role in organizing the salient content when generating summaries. For example, in Figure 1 the phrases Albert Einstein, the great prize and explanation of the of the pho-toelectric which distribute in different sentences are easily aggregated through their semantic relations to compose the final summary sentence. We further propose a graph-based encoder-decoder model based on the unified semantic graph. The graph-encoder effectively encodes long sequences by explicitly modeling the relations between phrases and capturing the global structure based on the semantic graph. Besides, several graph augmentation methods are also applied during graph encoding to tap the potential semantic relations. For the decoding procedure, the graph decoder incorporates the graph structure by graph propagate attention to guide the summary generation process, which can help select salient content and organize them into a coherent summary. We conduct extensive experiments on both the long-document summarization dataset BIGPATENT and MDS dataset WikiSUM to validate the effectiveness of our model. Experiment results demonstrate that our graph-based model significantly improves the performance of both long-document and multi-document summarization over several strong baselines. Our main contributions are summarized as follows:  We present the unified semantic graph which aggregates co-referent phrases distributed in context for better modeling the long-distance relations and global structure in long-document summarization and MDS. We propose a graph-based encoder-decoder model to improve both the document representation and summary generation process of the Seq2Seq architecture by leveraging the graph structure. Automatic and human evaluation on both long-document summarization and MDS outperform several strong baselines and validate the effectiveness of our graph-based model. Abstractive summarization aims to generate a fluent and concise summary for the given input document (Rush et al., 2015). Most works apply Seq2Seq architecture to implicitly learn the summarization procedure (See et al., 2017; Gehrmann et al., 2018; Paulus et al., 2017; Celikyilmaz et al., 2018). More recently, significant improvements have been achieved by applying pre-trained language models as encoder (Liu and Lapata, 2019b; Rothe et al., 2020) or pre-training the generation process leveraging a large-scale of unlabeled corpus (Dong et al., 2019; Lewis et al., 2020; Qi et al., 2020; Zhang et al., 2020a). In MDS, most of the previous models apply extractive methods (Erkan and Radev, 2004; Cho et al., 2019). Due to the lack of large-scale datasets, some attempts on abstractive methods transfer single document summarization (SDS) models to MDS (Lebanoff et al., 2018; Yang et al., 2019) or unsupervised methods based on auto-encoder (Chu and Liu, 2019; Brazinskas et al., 2020; Amplayo and Lapata, 2020). After the release of several large MDS datasets (Liu et al., 2018; Fabbri et al., 2019), some supervised abstractive models for MDS appear (Liu and Lapata, 2019a; Li et al., 2020). Their works also emphasize the importance of modeling cross-document relations in MDS. Explicit structures play an important role in recent deep learning-based extractive and abstractive summarization methods (Li et al., 2018a,b; Liu et al., 2019a). Different structures benefit summarization models from different aspects. Constituency parsing greatly benefits content selection Input Length 800 1600 2400 3000 #Nodes 140 291 467 579 #Edges 154 332 568 703 Table 1: Illustration of how the average number of nodes and edges in the graph changes when the input sequence becomes longer on WikiSUM. and compression for extractive models. Cao et al. (2015) propose to extract salient sentences based on their constituency parsing trees. Xu and Durrett (2019) and Desai et al. (2020) jointly select and compress salient content based on syntax structure and syntax rules. Dependency parsing helps summarization models in semantic understanding. Jin et al. (2020) incorporate semantic dependency graphs of input sentences to help the summarization models generate sentences with better semantic relevance . Besides sentence-level structures, document-level structures also attract a lot of attention. Fernandes et al. (2019) build a simple graph consisting of sentences, tokens and POS for summary generation. By incorporating RST trees, Xu et al. (2020) propose a discourse-aware model to extract sentences. Similarly, structures from semantic analysis also help. Liu et al. (2015) and Liao et al. (2018) propose to guide summarization with Abstract Meaning Representation (AMR) for a better comprehension of the input context. (Li and Zhuge, 2019) propose semantic link networks based MDS but without graph neural networks. Recently, the local knowledge graph by OpenIE attracts great attention. Leveraging OpenIE extracted tuples, Fan et al. (2019) compress and reduce redundancy in multi-document inputs in MDS. Their work mainly focus on the efficiency in processing long sequences. Huang et al. (2020) utilize OpenIE-based graph for boosting the faithfulness of the generated summaries. Compared with their work, our phrase-level semantic graph focus on modeling long-distance relations and semantic structures. In this section, we introduce the definition and construction of the unified semantic graph. The unified semantic graph is a heterogeneous graph defined as G = ( V, E ) , where V and E are the set of nodes and edges. Every node in V represents a concept merged from co-referent phrases. For example, in Figure 1 the node  Albert Einstein  is merged from phases  Albert Einstein  and  his  which indicate the same person by coreference resolution. Defined as a heterogeneous graph G , every node v  V and every edge e ij  E in our graph belongs to a type of phrase and dependency parsing relation, respectively. Determined by the type of phrases merged from, nodes are categorized into three different types: Noun phrase (N), Verb phrase (V), Other phrase (O). We neglect dependency relations in edges as they mainly indicate sentence syntax. Instead, the meta-paths (Sun et al., 2011) in the unified semantic graph convey various semantic relations. Notice that most O such as adjective phrases, adverb phrases function as modifiers, and the meta-path O-N indicates modification relation. The meta-path N-N between Noun phrases represents appositive relation or appositional relation. Furthermore, two-hop meta-path represents more complex semantic relations in graph. For example, N-V-N like [ Albert Einstein ]-[ won ]-[ the physics Nobel Prize ] indicates SVO (subjectverbobject) relation. It is essential to effectively model the two-hop meta-path for complex semantic relation modeling. To construct the semantic graph, we extract phrases and their relations from sentences by first merging tokens into phrases and then merging co-referent phrases into nodes. We employ CoreNLP (Man-ning et al., 2014) to obtain coreference chains of the input sequence and the dependency parsing tree of each sentence. Based on the dependency parsing tree, we merge consecutive tokens that form a complete semantic unit into a phrase. Afterwards, we merge the same phrases from different positions and phrases in the same coreference chain to form the nodes in the semantic graph. The final statistics of the unified semantic graph on WikiSUM are illustrated in table 1, which indicates that the scale of the graph expands moderately with the inputs. This also demonstrates how the unified semantic graph compresses long-text information. In this section, we introduce our graph-based abstractive summarization model, which mainly consists of a graph encoder and a graph decoder, as shown in Figure 2. In the encoding stage, our Albert Einstein was a theoretical physicist . model takes a document or the concatenation of a set of documents as text input (represented as x = { x k } ), and encodes it by a text encoder to obtain a sequence of local token representations. The graph encoder further takes the unified semantic graph as graph input (represented as G = ( V, E ) in section 3.1), and explicitly model the semantic relations in graph to obtain global graph representations. Based on several novel graph-augmentation methods, the graph encoder also effectively taps the implicit semantic relations across the text input. In the decoding stage, the graph decoder leverages the graph structure to guide the summary generation process by a novel graph-propagate attention, which facilitates salient content selection and organization for generating more informative and coherent summaries. To better represent local features in sequence, we apply the pre-trained language model RoBERTa (Liu et al., 2019b) as our text encoder. As the maximum positional embedding length of RoBERTa is 512, we extend the positional embedding length and randomly initialize the extended part. To be specific, in every layer, the representation of every node is only updated by it's neighbors by self attention. After we obtain token representations by the text encoder, we further model the graph structure to obtain node representations. We initialize node representations in the graph based on token representations and the token-to-node alignment information from graph construction. After initialization, we apply graph encoding layers to model the explicit semantic relations features and additionally apply several graph augmentation methods to learn the implicit structure conveyed by the graph. Node Initialization Similar to graph construction in section 3.2, we initialize graph representations following the two-level merging, token merging and phrase merging. The token merging compresses and abstracts local token features into higher-level phrase representations. The phrase merging aggregates co-referent phrases in a wide context, which captures long-distance and cross-document relations. To be simple, these two merging steps are implemented by average pooling. Graph Encoding Layer Following previous works in graph-to-sequence learning (Koncel-Kedziorski et al., 2019; Yao et al., 2020), we apply Transformer layers for graph modeling by applying the graph adjacent matrix as self-attention mask. Graph Augmentation Following previous works (Bastings et al., 2017; Koncel-Kedziorski et al., 2019), we add reverse edges and self-loop edges in graph as the original directed edges are not enough for learning backward information. For better utilizing the properties of the united semantic graph, we further propose two novel graph augmentation methods. Supernode As the graph becomes larger, noises introduced by imperfect graph construction also increase, which may cause disconnected sub-graphs. To strengthen the robustness of graph modeling and learn better global representations, we add a special supernode connected with every other node in the graph to increase the connectivity. Shortcut Edges Indicated by previous works, graph neural networks are weak at modeling multihop relations (Abu-El-Haija et al., 2019). However, as mentioned in section 3.1, the meta-paths of length two represent rich semantic structures that require further modeling the two-hop relations between nodes. As illustrated in Figure 2, in a N-V-N meta-path [ Albert Einstein ]-[ was ]-[ a theoretical physicist ], the relations [ Albert Einstein ]-[ was ] and [ was ]-[ a theoretical physicist ] are obviously less important than the two-hop relation [ Albert Einstein ][ a theoretical physicist ]. Therefore we add shortcut edges between every node and its two-hop relation neighbors, represented as blue edges in Figure 2. We have also attempted other complex methods such as MixHop (Abu-El-Haija et al., 2019), but we find shortcut edges are more efficient and effective. The effectiveness of these graph augmentation methods has also been validated in section 6.2. Token and node representations benefit summary generation in different aspects. Token representations are better at capturing local features while graph representations provide global and abstract features. For leveraging both representations, we apply a stack of Transformer-based graph decoding layers as the decoder which attends to both representations and fuse them for generating summaries. Let y l  1 t denotes the representation of t -th summary token output by ( l  1 )-th graph decoding layer. For the graph attention, we apply multi-head attention using y l  1 t as query and node representations V = { v j } as keys and values:  t,j = ( y l  1 t WQ )( v j WK ) T  d head (1) where WQ , WK  R d  d are parameter weights,  t,j denote the salient score for node j to y l  1 t . We then calculate the global graph vector g t as weighted sum over values of nodes: g t = (cid:80) j Softmax (  t,j )( v j WV ) where WV  R d  d is a learnable parameter. We also obtain contextu-alized text vector c t similar to the procedure above by calculating multi-head attention between y l  1 t and token representations. Afterwards, we use a graph fusion layer which is a feed-forward neural network to fuse the concatenation of the two features: d lt = W Td ([ g t , c t ]) , where W d  R 2 d  d is the linear transformation parameter and d lt is the hybrid representation of tokens and graph. After layer-norm and feed-forward layer, the l -th graph decoding layer output y lt is used as the input of the next layer and also used for generating the t th token in the final layer. Graph-propagate Attention When applying multi-head attention to graph, it only attends to node representations linearly, neglecting the graph structure. Inspired by Klicpera et al. (2019), we propose the graph-propagate attention to leverage the graph structure to guide the summary generation process. By further utilizing semantic structure, the decoder is more efficient in selecting and organizing salient content. Without extra parameters, the graph-propagation attention can be conveniently applied to the conventional multi-head attention for structure-aware learning. Graph-propagate attention consists of two steps: salient score prediction and score propagation. In the first step, we predict the salient score for every node linearly. We apply the output of multi-head attention  t  R | v | C in Equation 1 as salient scores, where | v | is the number of nodes in the graph and C is the number of attention heads. C is regarded as C digits or channels of the salient score for every node. We then make the salient score structure-aware through score propagation. Though PageR-ank can propagate salient scores over the entire graph, it leads to over-smoothed scores, as in every summary decoding step only parts of the content are salient. Therefore, for each node we only propagate its salient score p times in the graph, aggregating at most p -hop relations. Let  0 t =  t denotes the initial salient score predicted in previous step, the salient score after p -th propagation is:  pt =   A p  1 t + (1   )  0 t (2) where  A = AD  1 is a degree-normalized adjacent matrix of the graph 1 , and   (0 , 1] is the teleport 1 Adjacent matrix A contains self-loop and reverse edges. probability which defines the salient score has the probability  to propagate towards the neighbor nodes and 1   to restart from initial. The graph-propagation procedure can also be formulated as:  pt = (  p  A p + (1   )( p  1 (cid:88) i =0  i  A i ))  t (3) After p steps of salient score propagation, the graph vector is then calculated by weighted sum of node values: g (cid:48) t = (cid:88) j Softmax (  pt,j )( v j WV ) (4) where for the convenience of expression, the concatenation of multi-head is omitted. The output of fusing g (cid:48) t and c t is then applied to generate the t th summary token as mentioned before. In this section, we describe the datasets of our experiments and various implementation details. We evaluate our model on a SDS dataset and an MDS dataset, namely BIGPATENT (Sharma et al., 2019) and WikiSUM (Liu et al., 2018). BIGPATENT is a large-scale patent document summarization dataset with an average input of 3572.8 words and a reference with average length of 116.5 words. BIGPATENT is a highly abstractive summarization dataset with salient content evenly distributed in the input. We follow the standard splits of Sharma et al. (2019) for training, validation, and testing (1,207,222/67,068/67,072). WikiSUM is a large-scale MDS dataset. Following Liu and Lapata (2019a), we treat the generation of lead Wikipedia sections as an MDS task. To be specific, we directly utilize the preprocessed results from Liu and Lapata (2019a), which split source documents into multiple paragraphs and rank the paragraphs based on their titles to select top-40 paragraphs as source input. The average length of each paragraph and the target summary are 70.1 tokens and 139.4 tokens, respectively. We concatenate all the paragraphs as the input sequence. We use the standard splits of Liu and Lapata (2019a) for training, validation, and testing (1,579,360/38,144/38,205). We train all the abstractive models by max likelihood estimation with label smoothing (label smoothing factor 0.1). As we fine-tune the pretrained language model RoBERTa as text encoder, we apply two different Adam optimizers (Kingma and Ba, 2015) with  1 = 0 . 9 and  2 = 0 . 998 to train the pre-trained part and other parts of the model (Liu and Lapata, 2019b). The learning rate and warmup steps are 2e-3 and 20,000 for the pretrained part and 0.1 and 10,000 for other parts. As noticed from experiments, when the learning rate is high, graph-based models suffer from unstable training caused by the gradient explosion in the text encoder. Gradient clipping with a very small maximum gradient norm (0.2 in our work) solves this problem. All the models are trained for 300,000 steps on BIGPATENT and WikiSUM with 8 GPUs (NVIDIA Tesla V100). We apply dropout (with the probability of 0.1) before all linear layers. In our model, the number of graph-encoder layers and graph-decoder layers are set as 2 and 6, respectively. The hidden size of both graph encoding and graph decoding layers is 768 in alignment with RoBERTa, and the feed-forward size is 2048 for parameter efficiency. For graph-propagation attention, the parameter  is 0.9, and the propagation steps p is 2. During decoding, we apply beam search with beam size 5 and length penalty with factor 0.9. Trigram blocking is used to reduce repetitions. We evaluate the quality of generated summaries using ROUGEF 1 (Lin, 2004) and BERTScore (Zhang et al., 2020b). For ROUGE, we report unigram and bigram overlap between system summaries and reference summaries (ROUGE-1, ROUGE-2). We report sentence-level ROUGE-L for the BIGPATENT dataset and summary-level ROUGE-L for the WikiSUM for a fair comparison with previous works. We also report BERTScore 2 F 1 , a better metric at evaluating semantic similarity between system summaries and reference summaries. Results on MDS Table 2 summarizes the evaluation results on the WikiSUM dataset. We compare our model with several strong abstractive and extractive baselines. As listed in the top block, Lead and LexRank (Erkan and Radev, 2004) are two classic extractive methods. The second block shows the results of several different abstractive methods. TransS2S is the Transformer-based encoder-decoder model. By replacing the Transformer encoder in TransS2S with BERT (Devlin et al., 2019) or RoBERTa and training with two optimizers (Liu and Lapata, 2019b), we obtain two strong baselines BERTS2S and RoBERTaS2S. T-DMCA is the best model presented by Liu et al. (2018) for summarizing long sequence. HT is the best model presented by Liu and Lapata (2019a) with the hierarchical Transformer encoder and a flat Transformer decoder. GraphSum, presented by Li et al. (2020), leverages paragraph-level explicit graph by the graph encoder and decoder, which gives the current best performance on WikiSUM. We report the 2 We apply roberta-large L17 no-idf version as the metric model and rescale with baseline setting according to suggestions on https://github.com/Tiiiger/bert score. best results of GraphSum with RoBERTa and the input length is about 2400 tokens. The last block reports the results of our model BASS with the input lengths of 2400 and 3000. Compared with all the baselines, our model BASS achieves great improvements on all the four metrics. The results demonstrates the effectiveness of our phrase-level semantic graph comparing with other RoBERTa based models, RoBERTaS2S (without graph) and GraphSum (sentence-relation graph). Furthermore, the phrase-level semantic graph improves the semantic relevance of the generated summaries and references, as the BERTScore improvements of BASS is obvious. Results on SDS Table 3 shows our experiment results along with other SDS baselines. Similar to WikiSUM, we also report LexRank, TransS2S, and RoBERTaS2S. Besides, we report the performance of several other baselines. ORACLE is the upper-bound of current extrative models. Seq2seq is based on LSTM encoder-decoder with attention mechanism (Bahdanau et al., 2015). Pointer and Pointer+cov are pointer-generation (See et al., 2017) with and without coverage mechanism, respectively. FastAbs (Chen and Bansal, 2018) is an abstractive method by jointly training sentence extraction and compression. TLM (Pilault et al., 2020) is a recent long-document summarization method based on language model. We also report the performances of recent pretrianing-based SOTA text generation models BART (large) and Peagua-sus (base) on BIGPATENT, which both contain a parameter size of 406M . The last block shows the results of our model, which contains a parameter size of 201M . The results show that BASS consistently outperforms RoBERTaS2S, and comparable with current large SOTA models with only half of the parameter size. This further demonstrates the effectiveness of our graph-augmented model on long-document summarization. For a thorough understanding of BASS, we conduct several experiments on the WikiSUM test set, including the effects of the graph structure and input length. We also validate the effectiveness of the graph-augmentation methods in graph encoder and the graph-propagation attention in graph decoder by ablation studies. Graph Structure Analysis To analyze how the unified semantic graph benefits summarization learning, we conduct ablation studies on the graph structures. Illustrated in Table 4, after removing explicit relations between phrases by fully connecting all the nodes, the R-1 metric drops obviously which indicates the relations between phrases improve the informativeness of generated summaries. After further removing phrase merging, we observe a performance decrease in all the metrics, which indicates the long-distance relations benefit both the informativeness and fluency of summary. Ablation Study The experimental results of removing supernode and shortcut edges from the unified semantic graph prove the effectiveness of graph augmentation methods in the graph encoder. Experimental results without the gaph-propagation attention confirms that the structure of the unified semantic graph is also beneficial for decoding. Overall, the performance of the model drops the most when removing shortcut edges which indicates the rich potential information is beneficial for summarization. Finally, after removing all the graph-relevant components, performance dramatically drops on all the metrics. Length Comparison According to Liu et al. (2018), input length affects the summarization performance seriously for Seq2Seq models as most of them are not efficient at handling longer sequences. The basic TransS2S achieves its best performance at the input length of 800, while longer input hurts performance. Several previous models achieve bet-0.8k 1.6k 2.4k 3k 41.0 41.5 42.0 42.5 43.0 43.5 44.0 R-1 HTGSumBASS 0.8k 1.6k 2.4k 3k 26.5 27.0 27.5 28.0 28.5 R-2 0.8k 1.6k 2.4k 3k 35.0 35.5 36.0 36.5 37.0 37.5 38.0 R-L Figure 3: Comparison of HT, GraphSum (GSum in fig-ure), BASS under various length of input tokens. ter performance when utilizing longer sequences. As illustrated in Figure 3, the performance of HT remains stable when the input length is longer than 800. Leveraging the power of sentence-level graph, GraphSum achieves the best performance at 2,400 but its performance begins to decrease when the input length reaches 3000. Unlike previous methods, ROUGE-1 of BASS significantly increased in 3000 indicates that the unified semantic graph benefits salient information selection even though the input length is extreme. Abastractiveness Analysis We also study the abstractiveness of BASS and other summarization systems on WikiSUM. We calculate the average novel n-grams to the source input, which reflects the abstractiveness of a summarization system (See et al., 2017). Illustrated in Figure 4, BASS generates more abstract summaries comparing to recent models, GraphSum, HT, and weaker than RoBERTaS2S. Summarized from observation, we draw to a conclusion that RoBERTaS2S usually generates context irrelevant contents due to the strong pretrained RoBERTa encoder but a randomly initialized decoder that relays on the long-text input poorly. Graph-based decoders of BASS and GraphSum alleviate this phenomenon. In addition to the above automatic evaluations, we also conduct human evaluations to assess the performance of systems. Because the patent dataset BIGPATENT contains lots of terminologies and requires professional background knowledge for annotators, we select WikiSUM as the dataset for evaluations. As Wikipedia entries can be summarized in many different aspects, annotators will naturally favor systems with longer outputs. Thus we first filter instances that the summaries of different systems are significantly different in lengths 1 2 3 N-gram 0 10 20 30 40 50 60 70 80 % n o v e l n g r a m HT TransformerS2S GraphSum BASS RoBERTaS2S Reference Figure 4: Illustration of novel n-grams in generated summaries form different systems. and then randomly select 100 instances. We invite 2 annotators to assess the summaries of different models independently. Annotators evaluate the overall quality of summaries by ranking them taking into account the following criterias: (1) Informativeness : whether the summary conveys important and faithful facts of the input? (2) Fluency : whether the summary is fluent, grammatical, and coherent? (3) Succinctness : whether the summary is concise and dose not describe too many details? Summaries with the same quality get the same order. All systems get score 2,1,-1,2 for ranking 1,2,3,4 respectively. The rating of each system is averaged by the scores of all test instances. The results of our system and the other three strong baselines are shown in Table 6. The percentage of rankings and the overall scores are both reported. Summarized from the results, our model BASS is able to generate higher quality summaries. Some examples are also shown in the appendix. Specifically, BASS generates fluent and concise summaries containing more salient content compared with other systems. The human evaluation results further validate the effectiveness of our semantic graph-based model. In this paper, we propose to leverage the unified semantic graph to improve the performance of neural abstractive models for long-document summarization and MDS. We further present a graph-based encoder-decoder model to improve both the document representation and summary generation process by leveraging the graph structure. Experiments Model 1 2 3 4 Rating TransS2S 0.32 0.14 0.09 0.45  0 . on both long-document summarization and MDS show that our model outperforms several strong baselines, which demonstrates the effectiveness of our graph-based model and the superiority of the unified semantic graph for long-input abstractive summarization. Though remarkable achievements have been made by neural network-based summarization systems, they still do not actually understand languages and semantics. Incorporating language structures in deep neural networks as prior knowledge is a straightforward and effective way to help summarization systems, as proved by this work and previous works. This work was partially supported by National Key R&D Program of China (No. 2020YFB1406701) and National Natural Science Foundation of China (No. 61876009).
abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method abstain abstain objective abstain abstain abstain method objective objective objective result other other other other other other other other other other other other other other other other other other other other other other other other other method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain objective result method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method method abstain abstain abstain method method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other abstain other abstain method method abstain abstain abstain method abstain method method abstain method method method method abstain abstain abstain abstain method abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain other abstain result result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result objective result abstain objective abstain abstain other