Sequence-to-sequence neural networks have recently achieved great success in abstractive summarization, especially through fine-tuning large pre-trained language models on the downstream dataset. These models are typically decoded with beam search to generate a unique summary. However, the search space is very large, and with the exposure bias, such decoding is not optimal. In this paper, we show that it is possible to directly train a second-stage model performing re-ranking on a set of summary candidates. Our mixture-of-experts SummaReranker learns to select a better candidate and consistently improves the performance of the base model. With a base PEGASUS, we push ROUGE scores by 5.44% on CNN-DailyMail (47.16 ROUGE-1), 1.31% on XSum (48.12 ROUGE-1) and 9.34% on Reddit TIFU (29.83 ROUGE-1), reaching a new state-of-the-art. Our code and checkpoints will be available at https://github.com/ntunlp/ SummaReranker . In recent years, sequence-to-sequence neural models have enabled great progress in abstractive summarization (See et al., 2017; Lin et al., 2021). In the news domain, they have surpassed the strong LEAD-3 extractive baseline. With the rise of transfer learning since BERT (Devlin et al., 2019), leading approaches typically fine-tune a base pre-trained model that either follows a general text generation training objective like T5 (Raffel et al., 2019), BART (Lewis et al., 2020), ERNIE (Zhang et al., 2019b) and ProphetNet (Qi et al., 2021), or an objective specifically tailored for summarization like in PEGASUS (Zhang et al., 2020). Most of these sequence-to-sequence models are history-based, where an output sequence is represented as a sequence of decisions and the probabil-* Equal contribution. ity of the sequence is computed as a product of decision probabilities. This is also known as the autoregressive factorization. To transform the sequence of probabilities into summaries, beam search is commonly used. While auto-regressive decoding with beam search is simple and has many advantages, it can be difficult to encode global constraints such as grammaticality, coherence and factual consistency within this framework, properties that are believed to be useful in discriminating among candidate outputs. If the model starts decoding in a bad direction, mistakes might propagate, carry over the mistake of previous tokens to the generation of new ones, and the model has no way to know that it should adjust the decoding. Furthermore, these models are typically trained with teacher forcing (Williams and Zipser, 1989), which leads to an inherent discrepancy between training time and inference time known as the exposure bias problem (Bengio et al., 2015; Sun and Li, 2021). Decoding methods such as beam search maintain a list of topk best candidates, and output a single best one. In the case of beam search, candidates are sorted by decreasing log-probability, and the last ( k  1) hypotheses are discarded. However, these ( k  1) other hypotheses often contain considerably better sequences in terms of different evaluation measures. This observation holds over other decoding methods: diverse beam search (Vi-4504 jayakumar et al., 2016), top-k sampling (Fan et al., 2018) and top-p sampling (Holtzman et al., 2019). In Table 1, we illustrate this phenomenon with the oracle scores (maximum scores over the pool of candidates) for four popular decoding methods and five metrics on the CNN-DailyMail (Hermann et al., 2015) dataset with a PEGASUS model. The oracle ROUGE-1 scores are up to 10 points higher (+22.8%) than the top beam baseline. Moreover, oracle gains significantly increase when mixing several generation methods together, reaching an improvement of more than 13 ROUGE-1 points (+30.5%). Such a gap is larger than the progress made by research in the whole field of neural abstractive summarization in the last five years (Nal-lapati et al., 2016; Dou et al., 2021). This suggests that current abstractive models are not exploited to their full capacity, calling for better methods to identify the best summary candidate. Given this assessment, we investigate whether it is possible to train a second-stage summarization model which learns to select the best summary among a set of candidates obtained from a base model and with a decoding process, which itself can potentially involve a set of decoding methods (e.g., beam search variants). This way, the model would recover the gap that separates it with the oracle. This raises the question of what makes a summary candidate the optimal one? Admittedly, summarization has been an underconstrained task and its evaluation is complex and remains an active research area (Kryscinski et al., 2019; Fabbri et al., 2021; Koto et al., 2021). To build a flexible approach, we use a multi-task learning framework based on a mixture-of-experts architecture in order to optimize jointly over several measures. To design a robust re-ranker, we systematically explore the dimensions of summary re-ranking: base model, decoding process, and evaluation measure. Our system, named SummaReranker , is flexible and multi-task: it can be trained with any set of evaluation metrics. It is considerably less com-putationnally expensive to train than the single-stage summarization models that it is plugged on. We apply our system across three different datasets {CNN-DailyMail, XSum, Reddit TIFU} and two base models {PEGASUS, BART}. Optimizing ROUGE metrics leads to relative performance improvements from 1.31% to 9.34% depending on the dataset. It outperforms recently proposed second-stage summarization approaches RefSum (Liu et al., 2021) and SimCLS (Liu and Liu, 2021) and sets a new state-of-the-art on CNN-DailyMail and XSum (Narayan et al., 2018). We present extensive quantitative results coupled with a qualitative human evaluation. Re-ranking has been adopted in several branches of NLP for long. In syntactic parsing, Collins and Koo (2005) were the first to employ a re-ranker on the outputs of a base parser, followed by Charniak and Johnson (2005), who used a Maximum Entropy re-ranker. Passage re-ranking is used as the first stage of question-answering systems, to retrieve relevant passages where the answer might lay (Kratzwald and Feuerriegel, 2018; Nogueira and Cho, 2019). Some recent question-answering models also propose to perform answer re-ranking, to refine the answer selection (Kratzwald et al., 2019; Iyer et al., 2021). Re-ranking has also been used in neural machine translation. Checkpoint reranking (Pandramish and Sharma, 2020) generates several translation candidates with multiple model checkpoints, based on the observation (similar to the one we made in 1) that the oracle across checkpoints is of higher quality than just the last checkpoint. Bhattacharyya et al. (2021) use an energy-based model on top of BERT to select translation candidates with higher BLEU score. In abstractive summarization, second-stage approaches such as re-ranking remain underexplored. Recently, RefSum (Liu et al., 2021) defined a second-stage summarization framework which helps address the problem of the train-test distribution mismatch in second-stage models. With a base GSum model (Dou et al., 2021), the authors reach a 46.18 state-of-the-art ROUGE-1 on CNN-DailyMail. In SimCLS (Liu and Liu, 2021), the authors train a second-stage model with contrastive learning, using a ranking loss to select the best summary candidate from a pool of 16 diverse beam search candidates, reaching 46.67 ROUGE-1 on CNN-DailyMail. Our approach differs from RefSum and SimCLS in terms of model architecture and loss function, as well as summary candidate generation process. In contrast with RefSum, we use a single base model, but mix several decoding methods, as our goal is single-model improvement. Unlike SimCLS, we do not use a ranking loss, but directly model the probability that a summary candidate is the best one. To the best of our knowl-4505 edge, we are the first ones to propose a multi-task re-ranking system for abstractive summarization. This enables practitioners to leverage the recent rich literature in automatic abstractive summarization evaluation (Lin, 2004; Zhang et al., 2019a; Zhao et al., 2019a; Yuan et al., 2021). Our approach follows the paradigm of second-stage models. Specifically, given a source document S , a base model B , and a set of decoding methods D , we get a pool of m summary candidates C = { C 1 , . . . , C m } . Given an evaluation metric  in a set of metrics M , we get associated scores for each candidates S  = {  ( C 1 ) , . . . ,  ( C m ) } . Our goal is to train a model f  parameterized by  to explicitly identify the best summary candidate C   according to the metric, which is given by: C   = arg max C i  C {  ( C 1 ) , . . . ,  ( C m ) } (1) We frame this problem as a binary classification. C   is the positive candidate, while other candidates are treated as negative. For a metric  , the re-ranker f  is trained with a binary cross-entropy loss: L  =  y i log p  ( C i )  (1  y i ) log(1  p  ( C i )) (2) where y i = 1 if C i = C   , otherwise y i = 0 . Binary classification has been successfully employed for re-ranking in prior work (Nallapati, 2004; Nogueira and Cho, 2019). While multi-way classification could be an alternative, we noticed that for each generation method, a significant fraction of candidates share the same score for one or several metrics, while it is rare that all candidates share the same score (Appendix C-D). Thus, there is not enough signal to distinguish m candidates into m different classes, but enough for two classes. To optimize for N different metrics M = {  1 , . . . ,  N } simultaneously, we use a separate prediction head (tower) for each and we minimize the average over metric losses defined as: L = 1 N (cid:88)   ML  (3) 3.2 Model Architecture We first need to get a good representation of the summary candidate. To use contextual information, we concatenate the source with the candidate, Figure 1: SummaReranker model architecture , optimizing N metrics. separating the two with a special token: [ CLS ] Source [ SEP ] Candidate , and feed it to a pre-trained language model. In all experiments, we use RoBERTa-large (Liu et al., 2019) as encoder. Concatenating the source with the candidate enables RoBERTa to perform cross-attention between the two, which finds parts of the source relevant to the summary candidate. We take the [ CLS ] representation from RoBERTa's last layer, and feed it to a multi-layer perceptron (MLP). Once we have a joint representation of the source with the candidate (noted x ), we perform multi-task learning in order to optimize for the desired metrics. Since metrics are different, yet may be strongly correlated (e.g., ROUGE variants), we adopt a mixture-of-experts (MoE) architecture. In particular, we follow the sparse MoE approach (Shazeer et al., 2017), which introduces experts dropout. To adapt it to multi-task training, we use the multi-gate approach proposed in Zhao et al. (2019b). Given E experts E 1 , . . . , EE and N prediction towers T 1 , . . . , TN , the prediction for an input summary representation x for a metric  indexed by k  { 1 , . . . , N } is: f k ( x ) = T k ( E (cid:88) i =1 softmax ( W k x ) ( i ) E i ( x )) (4) where W k is the weight matrix associated with gate 4506 k . The corresponding prediction probability is: p  = sigmoid ( f k ( x )) (5) Experts are shared across all tasks, and through the softmax gates the model learns how much weight to assign to each expert for each task. Our SummaReranker model architecture is shown in Fig. 1. In practice, the shared bottom MLP consists in two fully-connected layers with ReLU activation (Glorot et al., 2011). Each expert E i is also a two-layer MLP with ReLU, and each prediction tower T k is a single-layer MLP. We set the number E of experts to be equal to twice the number of tasks ( N ), and the experts dropout to 50%, so that the effective number of experts being used during training matches N . Our model has 370.09 million trainable parameters, representing a slight 4.14% increase due to the mixture-of-experts compared to the off-the-shelf RoBERTa-large. Second-stage learning approaches may suffer from an inherent distribution bias. Indeed, the base model has a different output distribution on the training set than on the validation and test sets. Thus, it is ineffective to train a second-stage model on the training set outputs of the base model. To resolve this distribution shift, we shuffle the training set and randomly split it into equal parts, then fine-tune a pre-trained model on each half. Then, to build a training set for the re-ranker, we infer with each model on the half that it was not trained on. At testing time, we face two options:  Base setup : in this setup, we infer on the test set with one of the two base models trained on half the training set , then apply the re-ranker. Since the base models are trained on less data, their performance on the test set worsens. However, we will show that SummaReranker brings improvements which more than compensate this performance drop. Transfer setup : this setup consists in applying SummaReranker on top of a base model trained on the whole training set . Note that SummaReranker is still trained in the same fashion as before. There could be a distribution mismatch in this setting too, since SummaReranker needs to rank summary candidates of a potentially higher quality (generated by a model trained on the full data) than the summaries that it was trained on R-1 R-2 R-L BS BaS R-1 1.000 0.884 0.977 0.858 0.662 R-2 0.884 1.000 0.910 0.833 0.665 R-L 0.977 0.910 1.000 0.855 0.669 BS 0.858 0.833 0.855 1.000 0.682 BaS 0.662 0.665 0.669 0.682 1.000 Table 2: Pearson correlation coefficient between the five evaluation metrics {R-1, R-2, R-L, BS, BaS} for a base PEGASUS with beam search on CNN/DM . (generated by a model trained on half the data). Nevertheless, SummaReranker still transfers well and considerably improves the performance of the base model in this transfer setup. If D is made of multiple decoding methods {  1 , ...,  j } , each producing several candidates, the overall candidate set may be large, slowing down inference. Thus, to explore lower-resource inference setups, we separate the sets of decoding methods D train and D test used for training and inference, respectively, and enforce that D test  D train . Throughout our experiments, we vary all the three dimensions of our re-ranking framework: the base model B , the set of decoding methods D and the set of scoring metrics M . As base models, we use PEGASUS (Zhang et al., 2020) and BART (Lewis et al., 2020), each one in their large version, as they are leading summarization models with publicly available checkpoints. We obtain pre-trained and fine-tuned checkpoints from the HuggingFace transformers library (Wolf et al., 2020). For decoding methods ( D ), we experiment with beam search (referred to as 1 ), diverse beam search (2), topk sampling (3) and topp sampling (4). For each decoding method, we set the number of candidates to 15 , as it is close to the maximum which could fit in a standard 11GB RAM GPU when doing generation with PEGASUS-large. As set of metrics, we first use ROUGE (Lin and Hovy, 2003), in its commonly used three flavours of ROUGE-1 (noted R-1 ), ROUGE-2 (noted R-2 ) 4507 Dataset Domain # Data points # Words Train Val Test Doc. and ROUGE-L (noted R-L ) for summarization evaluation. We also leverage recently introduced model based evaluation methods BERTScore (noted BS ) (Zhang et al., 2019a) and BARTScore (noted BaS ) (Yuan et al., 2021), which both rely on contextual word embeddings from pre-trained language models. Thus, our total set of metrics is M = {R-1, R-2, R-L, BS, BaS}. As seen in Table 2, R-1 and R-L are strongly correlated (Pearson correlation score of 0.977). BARTScore is the least correlated to other metrics, suggesting that it captures aspects complementary to the other four. We train SummaReranker on the following datasets, covering multiple domains:  CNN-DailyMail (Hermann et al., 2015) contains 93k and 220k articles from the CNN and DailyMail newspapers, respectively. We use the non anonymized version from (See et al., 2017). XSum (Narayan et al., 2018) contains 227k articles from the BBC for years 2010 2017. While also in the news domain, XSum is by design significantly more abstractive than CNN/DM and is made of single-sentence summaries. Reddit TIFU (Kim et al., 2019) contains 120k posts from the popular online Reddit forum. As in other summarization works (Zhang et al., 2020), we use the TIFU-long subset, containing 37k posts. As there is no official split, we build a random 80:10:10 split for training:validation:test. We refer to Table 3 for statistics on each dataset. To help the model better discriminate between candidates, we found that sampling was useful. Specifically, during training, we rank candidates by decreasing sum of normalized scores for the evaluation metrics and keep the top m top and bottom m bottom candidates. Thus, training time varies in O ( m top + m bottom ) , while inference is in O ( m ) as we need to score each candidate. In practice, we found that taking m top = 1 and m bottom = 1 performed well, on top of decreasing the training time. This means that at training time, the model only sees two candidates per data point. We scale the pool of candidates that these two are sampled from to four decoding methods, totalling 60 summary candidates per source document. We train SummaReranker for five epochs. We use the Adafactor optimizer (Shazeer and Stern, 2018), with maximum learning rate 1e-5, warming up the learning rate linearly over the first 5% training steps. Training on CNN/DM takes four days on a single RTX 2080 Ti GPU. For inference, we need to output a single candidate. After getting predicted probabilities across each metric   M , we output the candidate maximizing the sum of predicted probabilities. Note that relaxing inference to allow for a different best candidate for each metric would improve performance, but is not practical. We perform inference with the model checkpoint maximizing the sum of the scores for the metrics on the validation set. First, we investigate how our model performs in the base setup described in 3. We apply SummaReranker on top of PEGASUS and BART models fine-tuned on each half. For each model, we decode using beam search (1) and diverse beam search (2). The latter performs better for PEGASUS, while the former is better for BART. We then apply SummaReranker optimized jointly for R-1, R-2, and R-L on 4508 Decoding methods Evaluation metrics Model Modelstage D train D test m OptimizedMetrics( M ) R-1 R-2 R-L BS BaS Gain(%) PEGASUS (Zhang et al., 2020) 1 {1} {1} 8 _ 44.16 21.56 41.30 _ _ _ PEGASUS our setup 1 {1} {1} 15 _ 44.23 21.48 41.21 87.39 -2.78 _ PEGASUS our setup 1 {2} {2} 15 _ 44.56 20.90 41.58 87.36 -2.81 _ BART (Lewis et al., 2020) 1 {1} {1} 5 _ 44.16 21.28 40.90 _ _ _ BART our setup 1 {1} {1} 15 _ 43.28 20.44 40.06 87.78 -2.48 _ BART our setup 1 {2} {2} 15 _ 44.48 21.21 41.60 88.11 -2.33 _ BART + R3F (Aghajanyan et al., 2020) 1 {1} {1} 5 _ 44.38 21.53 41.17 _ _ _ GSum (Dou et al., 2021) 1 {1} {1} 4 _ 45.94 22.32 42.48 _ _ _ GSum + RefSum (Liu et al., 2021) 2 {1} {1} 4 _ 46.18 22.36 42.91 _ _ _ BART + SimCLS (Liu and Liu, 2021) 2 {2} {2} 16 _ 46.67 22.15 43.54 66.14 _ _ PEGASUS + SR 2 {1} {1} 15 {R-1, R-2, R-L} 45.56  22.23  42.46  87.60  -2.74  3.18 PEGASUS + SR 2 {2} {2} 15 {R-1, R-2, R-L} 46.86  22.01  43.59  87.66  -2.73  5.10 PEGASUS + SR 2 {1, 2} {1} 15 {R-1, R-2, R-L} 46.13  22.61  42.94  87.67  -2.72  4.59 PEGASUS + SR 2 {1, 2} {2} 15 {R-1, R-2, R-L} 46.83  21.88  43.55  87.63  -2.74  4.84 BART + SR 2 {1} {1} 15 {R-1, R-2, R-L} 44.60  21.38  41.36  88.03  -2.40  3.63 BART + SR 2 {2} {2} 15 {R-1, R-2, R-L} 46.47  22.17  43.45  88.43  -2.19  4.48 BART + SR 2 {1, 2} {1} 15 {R-1, R-2, R-L} 45.08  21.79  41.85  88.13  -2.37  5.08 BART + SR 2 {1, 2} {2} 15 {R-1, R-2, R-L} 46.50  22.15  43.50  88.45  -2.18  4.51 PEGASUS + SR ( new SOTA ) 2 {1, 2} {1, 2} 30 {R-1, R-2, R-L} 47.16  22.55  43.87  87.74  -2.71  5.44 PEGASUS + SR 2 {1, 2} {1, 2} 30 {BS, BaS} 45.00  20.90 41.93  87.56  -2.55  4.23 PEGASUS + SR 2 {1, 2} {1, 2} 30 {R-1, R-2, R-L, BS, BaS} 46.59  22.41  43.45  87.77  -2.58  4.39 BART + SR 2 {1, 2} {1, 2} 30 {R-1, R-2, R-L} 46.62  22.39  43.59  88.47  -2.18  5.05 BART + SR 2 {1, 2} {1, 2} 30 {BS, BaS} 44.90  20.85 42.03  88.28  -2.05  6.11 BART + SR 2 {1, 2} {1, 2} 30 {R-1, R-2, R-L, BS, BaS} 45.96  21.79  43.01  88.44  -2.09  4.03 PEGASUS + SR 2 {1, 2, 3, 4} {1, 2, 3, 4} 60 {R-1, R-2, R-L} 47.04  22.32  43.72  87.69  -2.74  _ Table 5: Transfer setup results on CNN/DM . SummaReranker improves a base PEGASUS by 4.57% to 7.21% with 15 candidates, and 8.70% to 9.36% with 30 candidates. With BART, SummaReranker improves by 3.94% to 11.65% with 15 candidates, and 7.98% with 30 candidates. When using several decoding methods, we compare the re-ranker performance with the best baseline among decoding methods. Notably, with SummaReranker, PEGASUS and BART models trained on 50% of the training set now surpass their counterparts trained on the whole training set, achieving 46.19 R-1 with PEGASUS and 45.96 R-1 with BART. This is better than GSum (Dou et al., 2021), the best reported summarization model on CNN/DM. Next, we look at how SummaReranker performs in the transfer setup. That means, we apply it on top of PEGASUS and BART models fine-tuned on the entire dataset, using public checkpoints. We also include R3F (Aghajanyan et al., 2020) and GSum (Dou et al., 2021) in our single-stage model comparison. In terms of second-stage approaches, we compare SummaReranker with RefSum (Liu et al., 2021) and SimCLS (Liu and Liu, 2021). Note that SummaReranker is trained as usual, on the outputs of two base models each trained on 50%. We first optimize for ROUGE metric {R-1, R-2, R-L} with multi-task training on CNN/DM (Ta-ble 5). With two decoding methods, PEGASUS + SummaReranker sets a new state of the art on CNN/DM with 47.16 R-1, 22.55 R-2 and 43.87 RL, corresponding to gains of 2.60/1.65/2.29 R-1/2/L or +5.44% from our diverse beam search baseline. As expected, the relative gains in transfer setup are lower than in base setup. Next, we optimize model-based metrics, and note the difficulty in improving BERTScore, compared to BARTScore. Optimizing jointly ROUGE and model-based metrics improves all metrics, but does not match the results when training only ROUGE. Interestingly, performance gains saturate when adding two extra decoding methods (topk and topp sampling), despite gains in the oracle scores observed in Table 1. To assert statistical significance of performance gains, we perform a t-test between SummaReranker scores and scores from the base model with each of the decoding methods being used, and mark with  results where the p -value is smaller than 0.05 for all these decoding methods. softmax weights from the gates) for the model optimized on all five metrics in Fig. 2. Notably, some experts specialize in certain metrics (for instance, expert 0 on R-2 and expert 4 on R-L). Then, we apply SummaReranker on XSum and Reddit TIFU, as shown in Table 6. We train SummaReranker using the three ROUGE metrics {R-1, R-2, R-L} as objective, and decoding methods {beam search, diverse beam search} to generate the candidates. On XSum, SummaReranker improves a base PEGASUS with beam search candidates by 1.31%, setting a new state-of-the-art of 48.12/24.95/40.00 R-1/2/L. On Reddit TIFU, we improve a base PEGASUS with beam search and diverse beam search (30 candidates) by 9.34%, reaching 29.83/9.50/23.47 R-1/2/L, and a base BART with beam search by 4.22%, reaching 28.99/9.82/22.96 R-1/2/L. Across datasets, training on a combination of beam search and diverse beam search candidates is consistently effective. Beyond summary properties, we investigate the performance of re-ranking itself with rank-based evaluation measures. A perfect re-ranker should always single out the best summary from the rest, yielding oracle results. To evaluate how SummaReranker ranks the best summary, we compute the best summary candidate recall at different thresholds. Since several candidates might get the same metric scores (Appendix C), the best candidate recall at threshold k for the random uniform ranking baseline is not the standard R @ k = km anymore Figure 2: Expert utilization for a base PEGASUS with SummaReranker optimized with {R-1, R-2, R-L, BS, BaS} on CNN/DM , with 10 experts. where m best is the number of best candidates. Following Fig. 3, a PEGASUS with diverse beam search ranking of summary candidates (dashed lines) is not significantly better than the corresponding random baseline from eq. (6) (dot-ted lines) on CNN/DM and Reddit TIFU. However, it improves on it on XSum, confirming the observation made in Table 6 that it is harder to train a re-ranker on this dataset. On all three datasets, SummaReranker (solid lines) significantly pushes the recall at all thresholds. We note +14.90 abso-lute recall@5 improvement on CNN/DM (50.84 versus 35.94, indicated by the black arrow), +9.54 on XSum and +5.23 on Reddit TIFU. Lastly, we demonstrate that re-ranking improvements in quantitative metrics also translate to qualitatively better summaries. Fig. 4 shows an example of summary selected by SummaReranker, alongside its source document, ground-truth (reference) summary and output from the base model. SummaReranker is able to include a whole sentence which was missed by the base summary. We refer to Appendix K for full re-ranking demonstrations on each of the three datasets. We also conduct a human evaluation. We asked three different humans to evaluate 50 randomly sampled test summaries for each dataset. Human raters were graduate students with professional English proficiency (TOEFL scores above 100 out of 120). Humans were shown the source document alongside the top beam search summary from Figure 5: Human evaluation results on all three datasets. PEGASUS, and the corresponding summary candidate selected by SummaReranker. They were asked to choose which one they believe is more faithful. They could choose a tie, because in some cases the base summary and the re-ranked one are very similar, or even identical (Appendix I). In Fig. 5, we see that on average, humans are more likely to pick the SummaReranker candidate. Abstractiveness Given that we are not modifying the base model nor its training procedure, we analyze whether our re-ranking system favors more abstractive candidates. In Fig. 6, we display the percentage of novel n -grams for n in {1,2,3,4}, for a base PEGASUS with beam search (blue) and diverse beam search (purple) decoding, and when adding SummaReranker in both cases (green and red, respectively). As first raised in (See et al., 2017), summary candidates are much less abstractive than ground truth summaries on CNN/DM. Yet, our re-ranker selects more abstractive candidates 4511 Figure 6: Novel n -grams with PEGASUS, across all datasets and with beam search and diverse beam search. according to all n -grams metrics, even more so with diverse beam search, which is already more abstractive than beam search. This observation also holds on Reddit TIFU and XSum (other than 1-grams). XSum summary candidates are already almost as abstractive as the ground truth and it is harder to obtain significant abstractiveness gains through our re-ranking. Speed/Performance trade-off On top of base model training and candidate generation, SummaReranker inference cost is linear in the number of candidates. A single candidate takes on average 38ms to be scored. As seen in Table 5 and Table 6, the performance gains from mixing several decoding methods to generate summary candidates are not scaling consistently (all four decoding methods are not better than just beam search and diverse beam search). To provide more insights on the speed/performance trade-off, we show in Appendix J SummaReranker performance when randomly sub-sampling k  { 1 , . . . , 15 } candidates. On CNN/DM, re-ranking as few as two candidates is sufficient to improve on the baseline PEGASUS. On XSum, it needs three to eight, and on Reddit TIFU three to four. As a rule of thumb, it is better to score all candidates when possible, but six to eight candidates provide a good trade-off between speed and performance across datasets. Further Work To encode the source jointly with the summary candidate, we need to truncate the source to a fixed number of tokens. Thus, we are limited by the maximum context window of the language model encoder (512 in the case of RoBERTa-large). Applying SummaReranker to long-document summarization, such as scien-tific articles summarization (Cohan et al., 2018) would need better long-range modeling. In 3, we weighted metric-dependent losses uniformly. We leave to further work the exploration of more complex weight balancing or multi-task learning objectives (Lin et al., 2019). We introduced SummaReranker, the first multi-task re-ranking framework for abstractive summarization. Encoding the source with the candidate, our model predicts whether the summary candidate maximizes each of the metrics optimized for. SummaReranker works well across diverse datasets, models, decoding methods and summarization evaluation metrics. Summaries selected by SummaReranker improve the ROUGE state-of-the-art on CNN/DM and XSum. In addition, we also show that they are more abstractive and more likely to be preferred by human evaluators over base model outputs. This research was supported by the SINGA scholarship and partially supported by the National Research Foundation, Prime Minister's Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. We would like to thank anonymous reviewers for their insightful feedback on how to improve the paper.
abstain abstain abstain result result objective other abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain method objective method abstain method abstain abstain result other other other other other method other other other other other objective objective method objective other method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method method method method abstain other abstain abstain abstain abstain method method abstain abstain abstain method method method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain method abstain abstain method abstain abstain method method method method abstain method method abstain method abstain abstain method abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result method abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain method method abstain abstain abstain objective method abstain abstain result other other