We present a new neural model for text summarization that first extracts sentences from a document and then compresses them. The proposed model offers a balance that sidesteps the difficulties in abstractive methods while generating more concise summaries than extractive methods. In addition, our model dynamically determines the length of the output summary based on the gold summaries it observes during training, and does not require length constraints typical to extractive summarization. The model achieves state-of-the-art results on the CNN/DailyMail and Newsroom datasets, improving over current extractive and abstractive methods. Human evaluations demonstrate that our model generates concise and informative summaries. We also make available a new dataset of oracle compressive summaries derived automatically from the CNN/DailyMail reference summaries. 1 1 Introduction Text summarization is an important NLP problem with a wide range of applications in data-driven industries (e.g., news, health, and defense). Single document summarizationthe task of generating a short summary of a document preserving its informative content (Sparck Jones, 2007)has been a highly studied research topic in recent years (Nallapati et al., 2016b; See et al., 2017; Fan et al., 2018; Pasunuru and Bansal, 2018). (EXCONSUMM Extractive)  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen, the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. Ibrahim al-Rubaish died Monday night in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike. (EXCONSUMM Compressive)  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen , the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. have primarily focused on two strategies: extractive and abstractive . The former select a subset of the sentences to assemble a summary (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018a,c). The latter generates sentences that do not appear in the original document (See et al., 2017; Narayan et al., 2018b; Paulus et al., 2018). Both methods suffer from significant drawbacks: extractive systems are wasteful since they cannot trim the original sentences to fit into the summary, and they lack a mechanism to ensure overall coherence. In contrast, abstractive systems require natural language generation and semantic representation, problems that are inherently harder to solve than just extracting sentences from the original document. In this paper, we present a novel architecture that attempts to mitigate the problems above via a middle ground, compressive summarization (Martins and Smith, 2009). Our model selects a set of sentences from the input document, and compresses them by removing unnecessary words, while keeping the summaries informative, concise and grammatical. We achieve this by dynamically modeling the generated summary using a Long Short Term Memory (LSTM; Hochreiter and Schmidhuber, 1997) to produce summary state representations . This state provides crucial information to iteratively increment summaries based on previously extracted information. It also facilitates the generation of variable length summaries as opposed to fixed lengths, in previous extractive systems (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018c; Zhang et al., 2018). Our model can be trained in both extractive (labeling sentences for extraction) or compressive (labeling words for extraction) settings. Figure 1 shows a summary example generated by our model. Our contributions in this paper are three-fold:  we present the first end-to-end neural architecture for EXtractive and COmpressive Neural SUMMarization (dubbed EXCONSUMM , see 3),  we validate this architecture on the CNN/DailyMail and the Newsroom datasets (Hermann et al., 2015; Grusky et al., 2018), showing that our model generates variable-length summaries which correlate well with gold summaries in length and are concise and informative (see 5), and  we provide a new CNN/DailyMail dataset annotated with automatic compressions for each sentence, and a set of compressed oracle summaries (see 4). Experimental results show that when evaluated automatically, both the extractive and compressive variants of our model provide state-of-the-art results. Human evaluation further shows that our model is better than previous state-of-the-art systems at generating informative and concise summaries. Recent work on neural summarization has mainly focused on sequence-to-sequence (seq2seq) architectures (Sutskever et al., 2014), a formulation particularly suited and initially employed for abstractive summarization (Rush et al., 2015). However, state-of-the-art results have been achieved by RNN-based methods which are extractive. They select sentences based on an LSTM classifier that predicts a binary label for each sentence (Cheng and Lapata, 2016), based on ranking using reinforcement learning (Narayan et al., 2018c), or even by training an extractive latent model (Zhang et al., 2018). Other methods rely on an abstractive approach with strongly conditioned generation on the source document (See et al., 2017). In fact, the best results for abstractive summarization have been achieved with models that are more extractive in nature than abstractive, since most of the words in the summary are copied from the document (Gehrmann et al., 2018). Due to the lack of training corpora, there is almost no work on neural architectures for compressive summarization. Most compressive summarization work has been applied to smaller datasets (Martins and Smith, 2009; Berg-Kirkpatrick et al., 2011; Almeida and Martins, 2013). Other non-neural summarization systems apply this idea to select and compress the summary. Dorr et al. (2003) introduced a method to first extract the first sentence of a news article and then use linguistically-motivated heuristics to iteratively trim parts of it. Durrett et al. (2016) also learns a system that selects textual units to include in the summary and compresses them by deleting word spans guided by anaphoric constraints to improve coherence. Recently, Zhang et al. (2018) trained an abstractive sentence compression model using attention-based sequence-to-sequence architecture (Rush et al., 2015) to map a sentence in the document selected by the extractive model to a sentence in the summary. However, as the sentences in the document and in the summary are not aligned for compression, their compression model is significantly inferior to the extractive model. In this paper, we propose a novel seq2seq architecture for compressive summarization and demonstrate that it avoids the over-extraction of existing extractive approaches (Cheng and Lapata, 2016; Dlikman and Last, 2016; Nallapati et al., 2016a). Our model builds on recent approaches to neural extractive summarization as a sequence labeling problem, where sentences in the document are labeled to specify whether or not they should be included in the summary (Cheng and Lapata, 2016; Narayan et al., 2018a). These models often condition their labeling decisions on the document representation only. Nallapati et al. (2017) tries to model the summary as the average representation = 1 ... ... ...   ... of the positively labeled sentences. However, as we show later, this strategy is not the most adequate to ensure summary coherence, as it does not take the order of the selected sentences into account. Our approach addresses this problem by maintaining an LSTM cell to dynamically model the generated summary. To the best of our knowledge, our work is the first to use a model that keeps a state of already generated summary to effectively model variable-length summaries in an extractive setting, and the first to learn a compressive summarizer with an end-to end approach. Our model extracts sentences from a given document and further compresses these sentences by deleting words. More formally, we denote a document D = ( s 1 , . . . , s M ) as a sequence of M sentences, and a sentence s i = ( w i 1 , . . . , w iN ) as a sequence of N words. We denote by e ( w ij ) , e ( s i ) and e ( D ) the embedding of words, sentences and document in a continuous space. We model document summarization as a sequence labeling problem where the labeler transitions between internal states. Each state is dynamically computed based on the context, and it combines an extractive summarizer followed by a compressive one. First, we encode a document in a multi-level approach, to extract the embeddings of words and sentences (Document Encoder). Second, we decode these embeddings using a hierarchical De-cision Decoder. The extractive summarizer labels each sentence s i with a label z i  { 0 , 1 } where 1 indicates that the sentence should be included in the final summary and 0 otherwise. An extractive summary is then assembled by selecting all sentences with the label 1 . Analogously, the compressive summarizer labels each word w ij with a label y ij  { 0 , 1 } , denoting whether the word j in sentence i is included in the summary or not. The final summary is then assembled as the sequence of words w ij for each z i = 1 and y ij = 1 . See Figures 2 and 3 for an overview of our model. We next describe each of its components in more detail. The document encoder is a two layer biLSTM, one layer encoding each sentence, and the second layer encoding the document. The first layer takes as input the word embeddings e ( w ij ) for each word j in sentence s i , and outputs the hidden representa-SentStates Compressive decoder Extractive decoder WordStates Figure 3: Decision decoder architecture. tion of each word h wij . The hidden representation consist of the concatenation of a forward   h wij and a backward   h wij LSTM (WordEncoder in Figure 2). This layer eventually outputs a representation for each sentence e ( s i ) = [   h wiN ,   h wi 1 ] that corresponds to the concatenation of the last forward and first backward LSTMs. The second layer encodes information about the document and is also a biLSTM that runs at the sentence-level. This biLSTM takes as input the sentence representation from the previous layer e ( s i ) and outputs the hidden representation for each sentence s i in the document as h si (SentEncoder in Figure 2). We consider the output of the last forward LSTM over M sentences and first backward LSTM to be the final representation of the document e ( D ) = [   h sM ,   h s 1 ] . The encoder returns two output vectors, d si = [ e ( D ) , e ( s i ) , h si ] associated with each sentence s i , and d wij = [ e ( D ) , e ( s i ) , e ( w ij ) , h si , h wij ] for each word j at the specific state of the encoder i . Given that our model operates both at the sentence-level and at the word-level, the decision decoder maintains two state LSTMs denoted by SentStates and WordStates as in Figure 3. For the sentence-level decoder sentences are selected and the state of the summary gets updated by SentStates . For the word-level, all compressed word representations in a sentence are pushed to the word-level layer. In the compressive decoder, words that get selected are pushed onto the WordStates , and once the decoder has reached the end of the sentence, it pushes the output representation of the last state onto the sentence-level layer for the next sentence. Extractive Decoder The extractive decoder selects the sentences that should go to the summary. For each sentence s i at time step i , the decoder takes a decision based on the encoder representation d si and the state of the summary o si , computed as follows: o si = SentStates ({ e ( c k )} k < i,z k = 1 ) . where the o si is modeled by an LSTM taking as input the already selected and compressed sentences comprising the summary so far { e ( c k )} k < i,z k = 1 . This way, at each point in time, we have a representation of the summary given by the SentStates LSTM that encodes the state of summary generated so far, based on the past sentences already processed by the compressive decoder e ( c i  1 ) (in WordStates ). 2 The summary representation at step i ( o si ) is then used to determine whether to keep or not the current sentence in the summary ( z i = 1 or 0 respectively). The summarizer state subsumes information about the document, sentence and summary as: p i = tanh ( WE [ d si ; o si ] + b s ) , where WE is a model parameter, o si is the dynamic LSTM state, and b s is a bias term. This modeling decision is crucial in order to generate variable length summaries. It captures information about the sentences or words already present in the summary, helping in better understanding the true length of the summary given the document. Finally, the summarizer state p i is used to compute the probability of the action at time i as: p ( z i  p i ) = exp ( W z i p i + x z i )  z   { 0 , 1 } exp ( W z  p i + x z  ) , 2 When using only the extractive model the summary state o si is generated from an LSTM whose inputs correspond to the sentence encoded embeddings { e ( s k )} k < i,z k = 1 instead of the previously generated compressed representations { e ( c k )} k < i,z k = 1 . where W z is a model parameter and x z is a bias term for the summarizer action z . We minimize the negative log-likelihood of the observed labels at training time (Dimitroff et al., 2013), where  s 0 and  s 1 represent the distribution of each class for the given sentences: 3 L (  s ) =   c  { 0 , 1 }  sc M  i = 1 1 z i = c  i,z i = 0 log p ( z i  p i ) , where 1 z i = c is the indicator function of class c and  s represents all the training parameters of the sentence encode/decoder. At test time, the model emits probability p ( z i  p i ) , which is used as the soft prediction sequentially extracting the sentence i . We admit sentences when p ( z i = 1  p i ) > 0 . 5 . Compressive Decoder Our compressive decoder shares its architecture with the extractive decoder. The compressive layer is triggered every time a sentence is selected in the summary and is responsible for selecting the words within each selected sentence. In practice, WordStates LSTM (see Figure 3) is applied hierarchically after the sentence-level decoder, using as input the collected word embeddings so far: o wij = WordStates ({ e ( w ik )} k  j,y ik = 1 ) . After making the selection decision for all words pertaining to a sentence, the final state of the WordStates , e ( c i ) = o wiN is fed back to SentStates of the extractive level decoder for the consecutive sentence, as depicted in Figure 3. The word-level summarizer state representation depends on the encoding of words, document and sentence d wij , on the dynamic LSTM encoding for the summary based on the selected words ( WordStates ) o wij and sentences ( SentStates ) o si : q ij = tanh ( WC [ d wij ; o si ; o wij ] + b w ) , where WC is a model parameter and b w is a bias term. 3 If M   Mi = 1 z i = 0 or  Mi = 1 z i = 0 , we simply consider the whole term to be zero. Here M represents the number of sentences in the document. with parameter W y ij and bias x y ij . The final loss for the compressive layer is L (  w ) = M  i = 1 z i  ( i   w ) , where  w represents the set of all the training parameters of the word-level encoder/decoder,  ( i ) is the compressive layer loss over N words:  ( i   w ) =   c  { 0 , 1 }  wc M  i = 1 1 y ij = c  i,z i = 0 log p ( y ij  q ij ) . The total final loss is then given by the sum of the extractive and compressive counterparts, L (  ) = L (  s ) + L (  w ) . We mainly used the CNN/DailyMail corpus (Hermann et al., 2015) to evaluate our models. We used the standard splits of Hermann et al. (2015) for training, validation, and testing (90,266/1,220/1,093 documents for CNN and 196,961/12,148/10,397 for DailyMail). To evaluate the flexibility of our model, we also evaluated our models on the Newsroom dataset (Grusky et al., 2018), which includes articles form a diverse collection of sources (38 publishers) with different summary style subsets: extractive (Ext.), mixed (Mixed) and abstractive (Abs.). We used the standard splits of Grusky et al. (2018) for training, validation, and testing (331,778/36,332/36,122 documents for Ext., 328,634/35,879/36,006 for Mixed and 332,554/36,380/36,522 for Abs.). We did not anonymize entities or lower case tokens. Datasets for training extractive summarization systems do not naturally contain sentence/word-level labels. Instead, they are typically accompanied by abstractive summaries from which extraction labels are extrapolated. We create extractive and compressive summaries prior to training using two types of oracles . We used an extractive oracle to identify the set of sentences which collectively gives the highest ROUGE (Lin and Hovy, 2003) with respect to the gold summary (Narayan et al., 2018c). To build a compressive oracle , we trained a supervised sentence labeling classifier, adapted from Oracle R1 R2 RL Extractive Oracle 54.67 30.37 50.81 Compressive Oracle 57.12 32.59 53.27 Table 1: Oracle scores obtained for the CNN and DailyMail testsets. the Transition-Based Chunking Model (Lample et al., 2016), to annotate spans in every sentence that can be dropped in the final summary. We used the publicly released set of 10,000 sentence-compression pairs from the Google sentence compression dataset (Filippova and Altun, 2013; Filippova et al., 2015) for training. After tagging all sentences in the CNN and DailyMail corpora using this compression model, we generated oracle compressive summaries based on the best average of ROUGE-1 (R1) and ROUGE-2 (R2) F 1 scores from the combination of all possible sentences and all removals of the marked compression chunks. To verify the adequacy of our proposed oracles, we show in Table 1 a comparison of their scores. Our compressive oracle achieves much better scores than the extractive oracle, because of its capability to make summaries concise. Moreover, the linguistic quality of these oracles was preserved due to the tagging of the entire span by the sentence compressor trained on the sentence compression dataset. 4 We believe that our dataset with oracle compression labels will be of significant interest to the sentence compression and summarization community. The parameters for the loss at the sentence-level were  s 0 = 2 and  s 1 = 1 and at the word-level,  w 0 = 1 and  w 1 = 0 . 5 . We used LSTMs with d = 512 for all hidden layers. We performed mini-batch negative log-likelihood training with a batch size of 2 documents for 5 training epochs.We observed the convergence of the model between the 2nd and the 3rd epochs. It took around 12 hrs on a single GTX 1080 GPU to train. We evaluated our model on the validation set after every 5,000 batches. We trained with Adam (Kingma and Ba, 2015) with an initial learning rate of 0 . 001 . Our system was implemented using DyNet (Neubig et al., 2017). We evaluated summarization quality using F 1 ROUGE (Lin and Hovy, 2003). We report results 4 We show examples of both oracles in Appendix A.1. in terms of unigram and bigram overlap (R1) and (R2) as a means of assessing informativeness, and the longest common subsequence (RL) as a means of assessing fluency. 5 In addition to ROUGE, which can be misleading when used as the only means to assess summaries (Schluter, 2017), we also conducted a question-answering based human evaluation to assess the informativeness of our summaries in their ability to preserve key information from the document (Narayan et al., 2018c). 6 First, questions are written using the gold summary, we then examined how many questions participants were able to answer by reading system summaries alone, without access to the article. 7 Figure 5 shows a set of candidate summaries along with questions used for this evaluation. We evaluated our model EXCONSUMM in two settings: Extractive (selects sentences to assemble the summary) and Compressive (selects sentences and compresses them by removing unnecessary spans of words). We compared our models against a baseline (LEAD) that selects the first m leading sentences from each document, 8 three neural extractive models, and various abstractive models. For the extractive models, we used SUMMARUNNER (Nallapati et al., 2017), since it shares some similarity to our model, REFRESH (Narayan et al., 2018c) trained with reinforcement learning and LATENT (Zhang et al., 2018) a neural architecture that makes use of latent variable to avoid creating oracle summaries. We further compare against LATENT +C OMPRESS (Zhang et al., 2018), an extension of the LATENT model that learns to map extracted sentences to final summaries using an attention-based seq2seq model (Rush et al., 2015). All models, unlike ours, extract a fixed number of sentences to assemble their summaries. For abstractive models, we compare against the state-of-the art models of POINTER +C OVERAGE (See et al., 2017), ML+RL (Paulus et al., 2018), and Tan et al. (2017) among others. See Appendix A.2 for more details. 8 We follow Narayan et al. (2018c) and set m = 3 for CNN and 4 for DailyMail. We follow Grusky et al. (2018) and set m = 2 for Newsroom. Comparison with Extractive Systems. EXCONSUMM Compressive performs best on the CNN dataset and EXCONSUMM Extractive on the DailyMail dataset, probably due to the fact that the CNN dataset is less biased towards extractive methods than the DailyMail dataset (Narayan et al., 2018b). We report similar results on the Newsroom dataset. EXCONSUMM Compressive tends to perform better for mixed (Mixed) and abstractive (Abs.) subsets, while EXCONSUMM Extractive performs better for the extractive (Ext.) subset. Our experiments demonstrate that our compressive model tends to perform better on the dataset which promotes abstractive summaries. We find that EXCONSUMM Extractive consistently performs better on all metrics when compared to any of the other extractive models, except for the single case where it is narrowly behind LATENT on R2 (18.6 vs 18.8) for the CNN/DailyMail combined test set. It even outperforms REFRESH , which is trained with reinforcement learning. We hypothesize that its superior performance stems from the ability to generate variable length summaries. REFRESH or LATENT , on the other hand, always produces a fixed length summary. EXCONSUMM Compressive reports superior performance compared to LATENT +C OMPRESS (+4.2 for R1, +2.6 for R2 and +3.1 for RL). Our results demonstrate that our compressive system is more suitable for document summarization. It first selects sentences and then compresses them by removing irrelevant spans of words. It makes use of an advance oracle sentence compressor trained on a dedicated sentence compression dataset (Sec. 4.1). In contrast, LATENT +C OMPRESS naively trains a sequence-to-sequence compressor to map a sentence in the document to a sentence in the summary. Comparison with Abstractive Systems. Both EXCONSUMM Extractive and Compressive outperform most of the abstractive systems including Pointer+Coverage (See et al., 2017). When comparing with more recent methods (Pasunuru and Bansal, 2018; Gehrmann et al., 2018), our model has comparable performance. Summary Versatility. We evaluate the ability of our model to generate variable length summaries. Table 4 show the Pearson correlation coefficient between the lengths of the human generated summaries against each unbounded model. Our compressive approach obtains the best results, with a Pearson correlation coefficient of 0.72 ( p < 0 . 001 ). Figure 4 also shows the distribution of words Models Bounded Unbounded Human QA ROUGE Human QA ROUGE Pearson score rank R1 R2 RL score rank R1 R2 RL r LEAD 25.50 4 rd 30.9 11.9 29.1 36.33 5 th 31.6 13.5 29.3 0.40 REFRESH 20.88 6 th 37.4 17.3 34.8 66.34 1 st 43.8 25.8 41.6 0.60 LATENT 38.45 2 nd 38.9 19.6 36.4 53.38 4 th 40.7 22.0 38.1 -0.02 EXCONSUMM Extractive 36.34 3 rd 38.4 18.5 35.9 54.93 3 rd 40.8 21.0 38.2 0.68 EXCONSUMM Compressive 39.44 1 ST 38.8 19.0 37.0 57.32 2 nd 41.4 22.6 39.1 0.72 Pointer+Coverage 24.51 5 th 38.4 19.7 36.7 28.73 6 th 40.2 21.4 38.0 0.30 Table 4: QA evaluations: limited length (Bounded) and full length (Unbounded) summaries. per summary for the models where predictions were available. Interestingly, both EXCONSUMM Extractive and Compressive follow the human distribution much better than other extractive systems (LEAD , REFRESH and LATENT ), since they are able to generate variable-length summaries depending on the input text. Our compressive model generates a word distribution much closer to the abstractive Pointer+Coverage model but achieves better compression ratio; the summaries generated by Pointer+Coverage contain 59.8 words, while those generated by EXCONSUMM Compressive have 54.3 words on average. Table 4 shows results from our question answering based human evaluation. We elicited human judgements in two settings: the Unbounded, where participants were shown the full system produced summaries; and the Bounded, where participants were shown summaries that were limited to the same size as the gold summaries. For the Unbounded setting, the output summaries produced by REFRESH were able to answer most of the questions correctly, our Compressive and Extractive systems were placed at the 2nd and 3rd places respectively. 9 We observed that our systems were able to produce more concise summaries than those produced by REFRESH (avg. length in words: 76.0 for REFRESH , 56.2 for EXCONSUMM Extractive and 54.3 for EXCONSUMM Compressive; see Figure 4). REFRESH is prone to generating verbose summaries, consequently it has an advantage of accumulating more information. In the Bounded setting, we aim to reduce this unfair advantage. Scores are overall lower since the summary sizes are truncated to gold size. The EXCONSUMM Compressive summaries rank first and can answer 39.44% of questions correctly. EXCONSUMM Extractive retains its 3rd place answering 36.34% of questions correctly. 10 These results demonstrate that our models generate concise and informative summaries that correlate well with the human summary lengths. 11 5.3 Summary State Representation Next, we performed an ablation study to investigate the importance of the summary state representation o si w.r.t. the quality of the overall sum-9 We carried out pairwise comparisons between all models to assess whether system differences are statistically significant. We found that there is no statistically significant difference between REFRESH and EXCONSUMM Compressive. We use a one-way ANOVA with posthoc Tukey HSD tests with p < 0 . 01 . The differences among LATENT and both variants of EXCONSUMM , and between LEAD and Pointer+Coverage are also statistically insignificant. All other differences are statistically significant. 10 The differences among both variants of EXCONSUMM and LATENT , and among LEAD , REFRESH and Pointer+Coverage are statistically insignificant. All other differences are statistically significant. We use a one-way ANOVA with posthoc Tukey HSD tests with p < 0 . 01 . 11 App. A.2 shows more examples of our summaries. LEAD  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen, the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. Ibrahim al-Rubaish died Monday night in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike.  The Al-Malahem Media obituary characterized al-Rubaish as a religious scholar and combat commander. REFRESH  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen, the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. Ibrahim al-Rubaish died Monday night in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike.  Al-Rubaish was once held by the U.S. government at its detention facility in Guantanamo Bay, Cuba. LATENT  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen, the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. Ibrahim al-Rubaish died Monday night in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike. The Al-Malahem Media obituary characterized al-Rubaish as a religious scholar and combat commander. A Yemeni Defense Ministry official and two Yemeni national security officials not authorized to speak on record confirmed that al-Rubaish had been killed, but could not specify how he died. EXCONSUMM Extractive  (CNN) A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen, the terror group said, showing the organization is vulnerable even as Yemen appears close to civil war. Ibrahim al-Rubaish died Monday night in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike. EXCONSUMM Compressive  A top al Qaeda in the Arabian Peninsula leaderwho a few years ago was in a U.S. detention facilitywas among five killed in an airstrike in Yemen. Ibrahim al-Rubaish died in what AQAP's media wing, Al-Malahem Media, called a crusader airstrike. Pointer+Coverage  Ibrahim al-Rubaish was among a number of detainees who sued the administration of then-president George W. Bush to challenge the legality of their confinement in Gitmo. alRubaish was once held by the U.S. government at its detention facility in Guantanamo bay, Cuba. GOLD  AQAP says a crusader airstrike killed Ibrahim al-Rubaish  Al-Rubaish was once detained by the United States in Guantanamo Question-Answer Pairs  Who said that an airstrike killed Ibrahim al-Rubaish? mary. We tested against a STATE AVERAGING variant, where we replace o si by a weighted average, analogous to Nallapati et al. (2017), o avg s i =  j  1 i = 1 e ( s i ) p ( z i  p avgi ) , where p avgi has the same State ROUGE R1 R2 RL EXCONSUMM Extractive 32.5 12.6 28.5 STATE AVERAGING 30.0 12.3 26.9 EXCONSUMM Compressive 32.5 12.7 29.2 EXCONSUMM Ext+Comp oracle 25.5 9.3 23.7 Table 5: Summary state ablation for the CNN dataset. form as p i but depends recursively on the previous summary state o avg s i  1 . Table 5 shows that using an LSTM state o si to model the current sentences in the summary is very important. The other ablation study shows how learning to extract and compress in a disjoint approach (EXCONSUMM Ext+Comp oracle) performs against a joint learning approach (EXCONSUMM Compressive). We compared summaries generated from our best extractive model and compressed them with a compressive oracle. Our joint learning model achieves the best performance in all metrics compared with the other ablations, suggesting that joint learning and using a summary state representation is bene-ficial for summarization. We developed EXCONSUMM , a novel summarization model to generate variable length extractive and compressive summaries. Experimental results show that the ability of our model to learn a dynamic representation of the summary produces summaries that are informative, concise, and correlate well with human generated summary lengths. Our model outperforms state-of-the-art extractive and most of abstractive systems on the CNN and DailyMail datasets, when evaluated automatically, and through human evaluation for the bounded scenario. We further obtain state-of-the-art results on Newsroom, a more abstractive summary dataset. This work is supported by the EU H2020 SUMMA project (grant agreement N o 688139), by Lisbon Regional Operational Programme (Lis-boa 2020), under the Portugal 2020 Partnership Agreement, through the European Regional Development Fund (ERDF), within project INSIGHT (N o 033869), by the European Research Council (ERC StG DeepSPIN 758969), and by the Fundacao para a Ciencia e Tecnolo-gia through contracts UID/EEA/50008/2019 and CMUPERI/TIC/0046/2014 (GoLocal).
objective abstain result abstain objective objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method result abstain abstain method result other result result other other other other other other other other other other other other abstain method other other other abstain method objective method abstain abstain abstain other objective method other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other abstain other abstain abstain abstain abstain abstain abstain abstain other method abstain abstain other abstain method method method method method abstain abstain method method method abstain method method method method abstain method abstain abstain method method abstain method method abstain method method method abstain method method abstain method method method method abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain objective result result result other