Text summarization helps readers capture salient information from documents, news, interviews, and meetings. However, most state-of-the-art pretrained language models (LM) are unable to efficiently process long text for many summarization tasks. In this paper, we propose SUMMN , a simple, flexible, and effective multi-stage framework for input texts that are longer than the maximum context length of typical pretrained LMs. SUMMN first splits the data samples and generates a coarse summary in multiple stages and then produces the final fine-grained summary based on it. Our framework can process input text of arbitrary length by adjusting the number of stages, while keeping the LM input size fixed. Moreover, it can deal with both single-source documents and dialogues, and it can be used on top of different backbone abstractive summarization models. To the best of our knowledge, SUMMN is the first multi-stage split-then-summarize framework for long input summarization. Our experiments demonstrate that SUMMN outperforms previous state-of-the-art methods by improving ROUGE scores on three long meeting summarization datasets AMI, ICSI, and QMSum, two long TV series datasets from SummScreen, and a long document summarization dataset GovReport. Our data and code are available at https://github.com/ psunlpgroup/Summ-N . Abstractive summarization helps readers capture salient information from various sources such as documents, news, interviews, and meetings. Previous work has primarily focused on short texts of news (Gehrmann et al., 2018; Zhang et al., 2019) and short conversations (Gliwa et al., 2019; Chen and Yang, 2021). Recently proposed longer dialogue and document summarization tasks (Zhong et al., 2021b; Huang et al., 2021; Chen et al., 2021; Zhu et al., 2021a) pose challenges for current large pretrained language models due to the time and memory complexity of training, as well as limited input lengths these models can consume. A common method to handle long text reduces the input to a shorter one. This can be accomplished by truncating inputs (Lewis et al., 2020) or employing retrieve-then-summarize pipelines (Zhong et al., 2021b). However, these methods break the dependency of the context and decrease the number of tokens that the model can read, i.e., the receptive field of the model. The cutting-off model depends on the lead bias of the source text, while the retrieve-then-summarize models heavily rely on the independence of retrieved units (turns or sentences) which are usually scattered throughout the source text. Another approach optimizes the attention mechanism in Transformers to accommodate longer inputs by reducing the impact of quadratic complexity of the attention process using Locality-sensitive hashing (LSH) attention (Kitaev et al., 2020) and Sinkhorn attention (Tay et al., 2020). Additionally, HMNet (Zhu et al., 2020) and HAT-BART (Rohde et al., 2021) use hierarchical self-attention to extend the input limitation of typical self-attention models. However, the simplified attention mechanism weakens the power of pretrained Transformer models, e.g., HMNet is not pretrained on external large-scaled unsupervised datasets as BART did. In this paper, we propose SUMMN , a multi-stage framework for long dialogue and document summarization. Figure 1 shows the structure of SUMMN . First, it divides each source text into segments so that each can be completely fed into the backbone abstractive summarization model. Then, it matches each of them with the subset of target text using a ROUGE-based greedy algorithm. Next, each stage generates a coarse summary for each segment and concatenates them together as the input to the 1592 Input Segments TargetTargetTargetTargetTarget Match S u mm a r i z e r Finetune Inference S u mm a r i z e r Fine-grained Summary Target Segments Coarse Segments Coarse Summary Source Target Data Segmentation Coarse Summary Generation Fine-grained Summary Generation Text and Summary to form a pair of sample Model to generate summaries Original target Source Target Split Coarse Stage Fine-grained Stage Finetune Inference Figure 1: Workflow of the proposed SUMMN framework. next stage. After multiple stages of compression and summarization, the final stage produces a fine-grained summary. The process expands the model context to the full reception field, meaning that the proposed model can read the full input no matter how long the input is. Additionally, retrieve-then-summarize pipelines (Zhang et al., 2019) extract sentences individually, leading to the loss of the context information for understanding utterances. By contrast, SUMMN only cuts the source text at the end of each segment, so that the context of most sentences are retained. It does not assume lead bias because each part of the source is fully used. In addition, in each stage, it leverages a backbone abstractive summarization model to recursively generate the summaries. Therefore, it enjoys the full power of the pretrained language models because the framework preserves the intact structure of Transformers. SUMMN is flexible to inputs with different lengths by adjusting the number of stages. SUMMN can change the number of coarse stages according to the compression ratio between source and target, the input limit of the backbone model, and the input source length. We give the empirical formula to decide the number of needed stages for every tested dataset. Our experiments show that ROUGE increases on all datasets when increasing the number of stages from one to the appropriate number. Additionally, SUMMN is flexible because it can be applied to different backbone summarization models. For example, we found that the ROUGE scores increase sharply on the AMI dataset when replacing the backbone BART model with T5 (Raffel et al., 2020) and PEGASUS (Zhang et al., 2019). We conduct extensive experiments on long-input summarization datasets in multiple domains. The results demonstrate that the proposed model significantly outperforms previous state-of-the-art methods according to automatic and human evaluations on three long meeting summarization datasets (AMI, ICSI, QMSum) and one long TV series summarization dataset (SummScreen). It also achieves state-of-the-art performance on a long document summarization dataset (GovReport). These datasets include document summarization as well as both query-based and query-independent long dialogue summarization tasks. Our contributions are: (1) We propose SUMMN , a simple, flexible, and effective framework for long dialogue and document summarization. To the best of our knowledge, SUMMN is the first multi-stage split-then-summarize framework to solve long text summarization tasks. (2) We evaluate SUMMN on both dialogue and document domains and improve the baseline model by a large margin. (3) We analyze and compare the proposed framework with baselines and discuss its merits in detail. Long Document Summarization Long document summarization has been studied in multiple domains, such as news (Liu et al., 2021; Zhu et al., 2021b), patterns (Trappey et al., 2009), books (Kryscinski et al., 2021; Wu et al., 2021), sci-entific publications (Qazvinian and Radev, 2008; Mao et al., 2021), and medical records (Cohan et al., 2018). Gidiotis and Tsoumakas (2020) proposed a divide-and-conquer method by splitting the input into multiple segments, summarizing them separately, and combining the summary pieces. Grail et al. (2021) proposed a hierarchical neural model to process segmented input blocks. Compared with SUMMN , these models only split the input once, implying the lack of flexibility when handling longer input. The GovReport dataset was recently introduced containing documents with more than 9000 words, thus greatly challenging the capabilities of current models such as PEGASUS (Zhang et al., 2019), TLM (Pilault et al., 2020), and BIGBIRD (Zaheer et al., 2020). To handle this dataset, Huang et al. (2021) proposed head-wise positional strides to reduce the cost of the encoder-decoder attention. Similarly, models such as Longformer (Beltagy et al., 2020) and Reformer (Kitaev et al., 2020) adjust attention mechanisms in Transformers to consume longer inputs. However, these models sparsify the attention structure of the pretrained model to fit the longer source text. By contrast, SUMMN is able to maintain the full structure of various pretrained models. Long Dialogue Summarization Various models have also been proposed to handle long dialogue summarization. HMNet (Zhu et al., 2020) and HAT-BART (Rohde et al., 2021) leverage a two-level transformer-based model to obtain word level and sentence level representations. DialLM (Zhong et al., 2021a), Longformer-BART-arg (Fabbri et al., 2021) use finetuning or data augmentation to incorporate the external knowledge to maintain the accuracy of lengthy input. Different from these models, SUMMN is a framework without modifying the structure of the backbone attention model. Multi-Stage Text Generation Multiple multistage coarse-to-fine frameworks have been studied in many other text generation tasks, such as dialogue state tracking (Chen et al., 2020), neural story generation (Fan et al., 2018), and extractive summarization (Xu and Lapata, 2020). In a summarization task, a two-stage extract-and-summarize pipeline is commonly used (Zhang et al., 2019; Pilault et al., 2020; Zhao et al., 2020). However, unlike that work, our framework aims at long input summarization with fully abstractive intermediate summaries, meaning that SUMMN can be viewed as a summarize-then-summarize pipeline. Figure 1 shows the workflow of SUMMN . The workflow includes two types of stages, N coarse stages, and one fine-grained stage. Coarse stages include the data segmentation and coarse summary generation, while the fine-grained stage directly generates the summary as the final result. Besides, we have N + 1 separate models for each stage and each was separately trained. Our experiments show that the performance drops if different stages share the parameters (Section 4.2). SUMMN can adjust and compute the number of coarse stages N according to the stats of dataset and model. To formulate our task, we denote one sample of the source text as D = { D 1 , D 2 ,    , D m } , where D i indicates one sentence in a document or one turn in a dialogue. For query-based summarization, there is also a query Q . The goal is to generate a summary T , given D and the optional Q . In long text summarization, the number of tokens in the source data usually exceeds the limit of the backbone summarization models, thus reducing the quality of the summary. To make sure that the model can capture information about all source tokens, we apply a segmentation algorithm for long input summarization datasets. First, we segment the source text so that the data input to the backbone model does not exceed the length limit. Then, we apply a greedy algorithm to find the best target summary that matches the source segments. Source Segmentation Assume that the number of the maximum input tokens of the backbone model is K . To completely input the source information, we cut the input D (between sentences) into multiple segments, each of them containing fewer than K tokens. Given the input D , we will have n segments S = { S 1 , S 2 ,    , S n } where S i  D is a segment in D . For query-based summarization tasks, we simply concatenate the query to the beginning of the S , i.e. S i  Q (cid:76) S i . In both cases, the number of tokens in each segment is less than the hyper-parameter K . Target Matching Segmenting the source text results in n source pieces S i . We match each S i with a target segment T i  T to form the new pair ( S i , T i ) for the next step. We use a greedy algorithm for target matching. We first split T into separate sentences T s = { T s 1 , T s 2 ,    , T s k } . Then, each segment S i is matched with a subset of T s such that the ROUGE-1 score between the subset and S i is maximized. However, it is not feasible to find the optimal set due to the considerable running time. We apply a simple greedy approximation to find such a subset. From a null set T i , we iteratively add to the subset the sentence with the highest ROUGE-1 gain between T i and S i . Algorithm 1 shows how we obtain the new training pair ( S i , T i ) . (cid:76) indicates the concatenation of sentences while keeping them in the same order as in the original text. We use ROUGE-1 as the matching criterion because the higher ROUGE-1 score usually implies higher scores on the other metrics such as ROUGE-2 or ROUGE-L, while ROUGE-1 enjoys lower time complexity compared with other ROUGE metrics. This matching algorithm also ensures T i (cid:54) =  so that each S i can be matched to at least one target sentence. A sentence t  T s can be added to multiple subsets T i because one sentence of summary may need the information from multiple segments. In coarse summary generation, we train a summarization model, that takes the segmented data as input. We first collect the training samples ( S i , T i ) generated by data segmentation to form a new dataset. This augments the source data to d 1 /K times compared with the cut-off methods, where d 1 = | D 1 | indicates the averaged number of tokens of original source text. Thus, data segmentation helps the summarizer to better learn the task of the current stage. Additionally, because we incorporate the full input using segmentation, it does not rely on the leading bias in the cut-off method that only considers the first segment S 1 . Afterward, we use these data to train a neural summarizer. This way, our model treats each part of the source text as equally important. Given a source segment S i and an optional query Q , we obtain the coarse summary segments using a backbone summarization model: C li = SUMM l ( Q, S i ) Where l  [1 , N ] is the index of the current stage. Then, the n coarse summaries corresponding to the original source S = { S 1 , S 2 ,    , S n } are concatenated: C l = C l 1 (cid:76) C l 2 (cid:76)    (cid:76) C ln . We use C l as the new source text of next stage, which compresses the input source data D l . i.e. D l +1 = C l . To pair with the D l +1 , the target to the next stage is copied from the original dataset, i.e. T l +1 = T . The proposed framework is applicable to different backbone models SUMM l (  ) , such as BART (Lewis et al., 2020) and T5 (Raffel et al., 2020). We pick BART as the backbone model because it can best illustrate the benefits of our framework (Section 4.2). The number of stages can be estimated by data stats and model characteristics. In SUMMN , each coarse stage compresses the input to a shorter length. After N turns of coarse stages, the averaged length of source text is below K , the dataset is then fed into the fine-grained stage. Hence, the number of coarse stages can be computed by the following equation (details can be found in Appendix A):  N = (cid:100) log K  log d 1 log c 1  log K (cid:101) where d 1 and c 1 are the average length of source text and coarse segments in stage 1. In Section 5.7 and Table 9, we demonstrate this estimation is close to the empirical number of coarse stages. The greedy algorithm in SUMMN for target matching is critical to the performance. Consider a duplication algorithm where each segment S i is simply paired with the target T , i.e. T i = T . Since the target text is longer than the text segmented by Algorithm 1, the generated summary of each coarse stage will be longer as well, leading to a lower compression speed and larger N . Besides, 1595 Dataset Type Domain Size Source length Target length Query N + 1 AMI Dialogue Meetings 137 6007.7 296.6 (cid:55) 2 ICSI Dialogue Meetings 59 13317.3 488.5 (cid:55) 3 QMSum Dialogue Meetings 1808 9069.8 69.6 (cid:51) 2 SummScreen Dialogue TV shows 26851 6612.5 337.4 (cid:55) 2 GovReport Document Reports 19466 9409.4 553.4 (cid:55) 3 Table 1: The summarization datasets for evaluation. the duplication of the target will confuse the model, because some source segments will probably be paired with the same target, causing the model to generate duplicated content. Experiments (Table 7, stage 2 versus stage 2 & tar. seg.) show that ROUGE scores declines a lot when greedy target segment is replaced by the duplication algorithm . When the input source of D l is shorter than K we can proceed to the fine-grained stage. In this stage, D l is used to train a summarization model from scratch to obtain the final summary. The fine-grained stage works the same way as the vanilla backbone model. In fact, SUMMN with N = 0 is the backbone summarizer. In the fine-grained stage, the model is directly trained on dataset ( DN , T ) from the last coarse stage, and obtain the summary as the final output of SUMMN : F = SUMMN +1 ( Q, DN ) It is worth noting that, although source text may be shorter than 2 segments, i.e. d i  K , we still add them in all stages, so that each summarization model can be trained on the full dataset. We first list the datasets and metrics to evaluate the model. Then, we introduce the backbone model and baselines for comparisons. Finally, we present some implementation details. AMI & ICSI (McCowan et al., 2005; Janin et al., 2003) are meeting scripts generated by Automatic Speech Recognition (ASR) systems. AMI is collected from product design meetings in a company while ICSI is collected from academic group 1 Both QMSum and SummScreen can be accessed through SummerTime (Ni et al., 2021). meetings. Because the transcript is produced by ASR, there is a word error rate of 36% for AMI and 37% for ICSI. QMSum (Zhong et al., 2021b) is a query-based meeting summarization dataset. It consists of meetings from three domains, including AMI and ICSI, and the committee meetings of the Welsh Parliament and the Parliament of Canada. Each query and sample are written by experts. SummScreen (Chen et al., 2021) consists of community-contributed transcripts of television show episodes from The TVMegaSite, Inc. (TMS) and ForeverDream (FD). The summary of each transcript is the recap from TMS, or a recap of the FD shows from Wikipedia and TVMaze. GovReport (Huang et al., 2021) is a large-scale long document summarization dataset with 19,466 long reports published by the U.S. Government Accountability Office on national policy issues. We use ROUGE (Lin, 2004) as the automatic evaluation metric. 2 We split summary outputs into sentences to calculate the ROUGE-L score. If not specified, F1 scores are used in all results. We pick BART (Lewis et al., 2020) as our backbone summarization model because it performs well on short text summarization but not as good on longer texts, illustrating the benefits of our framework. Compared with other pretrained parameters, the BART-large model pretrained on the CNN/DM dataset yields the best performance (Zhang et al., 2021). So we use the BART-large-cnn parameter as a better starting point. It is worth noting that we use separate backbone models for each stage and each was separately trained. We experimented with reusing the model parameters in multiple stages but obtained a lower 2 We use pyrouge , a Python wrapper for the ROUGE: https://github.com/bheinzerling/pyrouge 1596 AMI ICSI QMSum-All QMSum-Gold R-1 R-2 R-L R-1 R-2 R-L R-1 R-2 R-L R-1 R-2 R-L PGNet 42.60 14.01 22.62* 35.89 6.92 15.67* 28.74 5.98 25.13 31.52 8.69 27.63 TopicSeg 51.53 12.23 25.47* ----HMNET 52.36 18.63 24.00* 45.97 10.14 18.54* 32.29 8.67 28.17 36.06 11.36 31.27 TextRank 35.19 6.13 16.70* 30.72 4.69 12.97* 16.27 2.69 15.41 --HAT-BART 52.27 20.15 50.57 43.98 10.83 41.36 ---DDAMS 53.15 22.32 25.67* 40.41 11.02 19.18* ---SUMMN 53.44 20.30 51.39 45.57 11.49 43.32 34.03 9.28 29.48 40.20 15.32 35.62 Table 2: ROUGE scores on three meeting summarizing tasks, AMI, ICSI, and QMSum. score, e.g. the ROUGE-1 score of stage 2 on the QMSum dataset decreases around two points if we use the best parameters of stage 1 summarizer as the starting point of training stage 2 summarizer. This is because the tasks of the different stages differ significantly. For instance, the input to the first stage of dialogue summarization is the dialogue turn while the input to the latter stages is the document. We compare the proposed framework with various baselines. PGNet (See et al., 2017) uses a pointer mechanism to copy the token from the training sample. TopicSeg (Li et al., 2019) is a multi-modal model jointly learning the segmentation and summarization. HMNet (Zhu et al., 2020) uses a hierarchical attention structure and cross-domain pre-training for meeting summarization. TextRank (Mihalcea and Tarau, 2004) is a graph-based ranking model for text processing. HAT-BART (Rohde et al., 2021) is a new hierarchical attention transformer-based architecture that outperforms standard Transformers. DDAMS (Feng et al., 2021) uses a relational graph to model the interaction between utterances by modeling different discourse relations. For the SummScreen dataset, we use the neural and hybrid model scores reported by Chen et al. (2021). We rename these two baselines as Long-former+ATT and NN+BM25+Neural to clarify the difference between other baselines. The baseline scores we report on GovReport are from the original paper (Huang et al., 2021). BART Variant indicates self-attention variants with full attention. BART HEPOS indicates encoder variants with head-wise positional strides (HEPOS) encoder-decoder attention. We fit all models into a single RTX A6000 GPU with a 48 GiB memory. We adopt the fairseq 3 implementation for BART. The learning rate is set to 2e-5 and the beam width is set to 2 for coarse stages and 10 for fine-grained stages. The maximum number of tokens in each batch is set to 2048. The maximum number of tokens in each source text is set to 1024 because we tried to extend the positional embeddings to 2048 or longer but obtained worse performance. We stop the coarse stage and start the fine-grained stage when the averaged source length is shorter than 2048 rather than 1024 to obtain a better performance (Section 5.7). For the output of each intermediate stage, we use <s> and </s> to separate each generated target segments C li . We discuss the evaluation results and effects of each component of SUMMN in this section. Meeting Summarization Table 2 shows the ROUGE scores on AMI, ICSI, and QMSum. Compared with the baseline models, SUMMN achieves state-of-the-art results on almost all metrics. Specifically, SUMMN improves SOTA on ICSI by 0.83 , and 1.96 ROUGE-2/L scores, improves SOTA on QMSum-Gold by 4.14 , 3.96 , and 4.35 ROUGE-1/2/L scores. These results demonstrate the effectiveness of SUMMN on long dialogue summarization tasks. TV Series Summarization Table 3 shows ROUGE scores on SummScreen. SUMMN outperforms on almost all metrics on two SummScreen datasets. Specifically, we improve 6.58 , 1.65 , and 3.75 ROUGE-1/2/L scores on the SummScreen-FD dataset. This result demonstrates the generalizability of SUMMN over various domains including meetings and TV series. Document Summarization Table 4 shows ROUGE scores on GoveReport. SUMMN achieves state-of-the-art performance on ROUGE-2 and ROUGE-L, and compatible results on ROUGE-1. The results show that SUMMN is applicable to both long dialogue and document summarization tasks. We also notice that the performance increases consistently when the number of stages goes up until the predefined number of stages. Figure 2 shows the ROUGE-1 scores of different tasks across stages. Stage 1 indicates the model with only one coarse stage and no fine-grained stage. In this model, We directly use the first segment of the coarse summary as the output, i.e. C 11 of each sample. Stage i ( i > 1 ) model contains i  1 coarse stages and one fine-grained stage, the generated 25 30 35 40 45 50 55 60 ROUG1 S c o r e Datasets Stage 1 Stage 2 Stage 3 Figure 2: ROUGE-1 scores of various datasets at different stages. Although stage 2 of SUMMN on the ICSI dataset has already outperformed the baselines, the scores can be further improved by adding one more coarse stage. In fact, on all datasets, increasing the number of stages leads to a performance gain. This gain can be explained as the following: if the output of the current stage is longer than K tokens, adding one more coarse stage will help since the model will receive more information from the source text compared with simply truncating them. On the contrary, if the input is smaller than K , there is no need to add more stages, because there is only one segment. SUMMN also boosts the performance of a backbone model by a large margin. As shown in Table 5, it improves the BART-large model by 6.87 , 3.89 , 6.78 ROUGE-1/2/L on AMI. This indicates the capability of SUMMN to boost the performance of a weak learner on long summarization tasks. In particular, when the backbone model is well pretrained on short input texts and performs well on short summarization tasks, SUMMN could greatly increase the capability of the backbone model to process and read long source texts. Also, the backbone of SUMMN can be easily replaced by some other models, and models do not necessarily have to be identical at every stage. For example, one can try different learners such as T5 as the backbone model and replace the model in stage 1 with a dialogue-to-document model. To demonstrate our framework can generalize to different backbone summarization models, we replace the BART-large-cnn model in previous experiments with other neural summarization models including T5 (Raffel et al., 2020) and PEGASUS (Zhang et al., 2019) using Hugging Face. Table 6 shows the ROUGE scores of three different models that are trained and evaluated on AMI. In all models, SUMMN improves the performance of backbone models by a large margin. For instance, although BART-base is a weaker summarizer compared with the BART-large model, the framework is still able to improve the ROUGE-1 score by 5.06. Table 7 shows the ablation study results of SUMMN on the AMI test set. Removing stage 2 (using the first segment of the coarse summary C 11 as the generated summary) leads to a 5.23 ROUGE-1 score drop. Without data segmentation, the ROUGE-1 score decreases by 6.61 using the same fine-grained stage. Removing both stage 2 and target matching (use duplication algorithm instead) further decreases the performance. It even hurts the perfor-R-1 R-2 R-L SUMMN 53.44 20.30 51.39 stage 2 48.21 18.59 46.46 data seg. mance of the original BART model because the duplication of targets will introduce some biases towards the common part of the targets. We conduct a human evaluation to assess the following: Readability takes into account word and grammatical error rate to evaluate how fluent the summary language is; Conciseness measures how well the summary discards the redundant information; Coverage measures how well the summary covers each part of the dialogue. We compare the results of SUMMN and HMNet because HMNet is a baseline model with the good capability to read whole input. For each meeting in AMI and ICSI dataset, we ask 3 different annotators with English expertise to label the summaries. Each annotator was asked to read the meeting transcript, gold summaries, and generated summaries using the SummVis (Vig et al., 2021) toolkit. They were asked to rate each summary from 1 to 5 (higher is better) for each metric. We also shuffle the summaries of two models to reduce the bias. Table 8 shows that SUMMN achieves higher scores in Readability , Conciseness , and Coverage than HMNet in both AMI and ICSI dataset. Specifi-cally, the Readability of SUMMN greatly surpasses the baseline by around 0.5/1 point on AMI/ICSI dataset. This is because BART is well-pretrained and is able to generate more readable text and SUMMN successfully maintains this capability. To gain more understanding of the multi-stage mechanism of SUMMN , we analyze the number of coarse stages and the compression rate through statistics of intermediate stages. Early Stopping of the Coarse Stage Although the ideal input of the final fine-grained stage should be shorter than K , the experiment results show that compressing input from 2 K to 1 K tokens usually hurts the performance of the model. This is probably because generating too many short segments which are hard to summarize confuses the model. Thus, we increase the length of input to the final fine-grained stage from K to 2 K to prevent noises in the training set. The modified formula to estimate the number of coarse stages  N is shown as follows (details in Appendix A). N val = 1 + log K  log d 1 log c 1  log K  N = (cid:100)  N val (cid:101) Number of Coarse Stages To verify that our estimation  N is close to the empirical number of coarse stages N , we use GovReport to compare the two as shown in Table 9. We choose this dataset because it contains the most number of samples among all five datasets, with completely three coarse stages as well. Table 9 shows the empirical/estimated number of coarse stages. To clearly show the  N value, we display the float number  N val as the estimated number, and N as the empirical number of re-maining coarse stages (Table 1). As can be seen, N =  N = (cid:100)  N val (cid:101) holds for all stages, meaning that the estimated  N is capable of estimating the correct N value. It is worth noting that, for stage 2 and stage 3, using this formula can also estimate how many additional coarse stage do we need. Compression Rate We analyze the change of compression rate across different stages. In SUMMN , compression rate R i is defined as the averaged source length of stage i divided by source length of stage i  1 . As shown in Table 9, both compression rates in stage 2 and stage 3 of GovReport are around 0.4, this shows that the compression rate of SUMMN across different stages are stable, meaning that the number of segments will decrease to around 40% of the previous stage steadily. Table 10 shows the time cost of inferring one sample using vanilla transformer versus SUMMN . Although the SUMMN needs to generate more tokens due to multi-stage pipeline, SUMMN reduces the inference time from quadratic to lower, i.e., from O ( n 2 ) to O ( Cn ) , C = K/ (1  R ) . Regarding training the model, SUMMN also need to infer O ( n ) additional tokens on the train/dev/test sets (details in Appendix B). In this paper, we propose SUMMN , a simple, flexible, and effective framework for long dialogue and document summarization. It consists of multiple coarse stages and one fine-grained stage to iteratively compress the long source input. It enjoys the full power of backbone models while ensuring the full receptive field of the summarization model. We evaluate the model on various datasets and improve the baselines by a large margin. The authors would like to thank Tao Yu, Ming Zhong, Yixin Liu, and Asli Celikyilmaz for their valuable discussions. We also would like to thank the anonymous reviewers for their helpful comments. This work is supported in part by a grant from Microsoft Research.
abstain abstain objective abstain abstain abstain objective objective other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result method abstain abstain abstain objective objective result objective other other other other other other other other other other other other other other other other objective abstain abstain abstain method method abstain method abstain abstain abstain method method method abstain method method method abstain abstain method method method abstain abstain method method method abstain method abstain abstain method method abstain abstain method method method method abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain method method other method abstain abstain method abstain abstain abstain abstain abstain abstain method method method abstain abstain method method abstain abstain method method method result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain result other other other