Extractive summarization methods operate by ranking and selecting the sentences which best encapsulate the theme of a given document. They do not fare well in domains like fictional narratives where there is no central theme and core information is not encapsulated by a small set of sentences. For the purpose of reducing the size of the document while conveying the idea expressed by each sentence, we need more sentence specific methods. Telegraphic summarization, which selects short segments across several sentences, is better suited for such domains. Telegraphic summarization captures the plot better by retaining shorter versions of each sentence while not really concerning itself with grammatically linking these segments. In this paper, we propose an unsupervised deep learning network (NUTS) to generate telegraphic summaries. We use multiple encoderdecoder networks and learn to drop portions of the text that are inferable from the chosen segments. The model is agnostic to both sentence length and style. We demonstrate that the summaries produced by our model show significant quantitative and qualitative improvement over those produced by existing methods and baselines.
The authors consider the problem of telegraphic sentence compression: they train a system in an unsupervised fashion to predict which words can be dropped from a sentence without drastic loss of information. To that end, they propose a new auto-encoding type architecture which uses the extracted words as latent code, and, most importantly, a linkage loss which relates a word's perplexity given the summary of its left context to its likelihood of being retained. The model itself is sober and well motivated, and the linkage loss is, to the best of my knowledge, original. The authors show that their method outperforms some simple baselines in terms of ROUGE and compression on a small human-annotated test set.
The paper is generally well written, although the initial presentation of the model could be made a little clearer (it is not obvious from the text that the Decoder takes the text as input -- Figure 2 helps, but comes a couple pages later). However, the authors fail to appropriately justify the choice of their hyper-parameters (e.g. "The optimum value of r for our experiments was found to be 0.65", "the best value of b was found to be 5", "The weights λ1, λ2, λ3, and λ4 have been set to 3, 2, 50 and 3 respectively for our experiments" -> how is "best" measured on the validation set, which does not have gold references?). The choice of the specific sparsity constraint (one could as well imagine using a simpe L1 regularization for the Binarization loss) and of \Chi_i (why not simply use the likelihood?) could also be better motivated.
The model also relies on a hand-crafted rules (Section 3.3) whose effect needs to be made more evident. What weights are used in practice? How were they chosen ("We observed that..." needs to be further developed)? The authors claim that "the quantitative scores are not affected significantly", but that is presumably only the ROUGE score, what about annotator's preferences?
Most importantly, however, the task of telegraphic sentence compression, whose usefulness is not a priori obvious, is barely motivated.  The author refer to "Malireddy et al. (2018)" for a justification, but it is important to note that the latter provides a telegraphic summary of a whole document, with a compression factor of 0.37. The claim is that the concatenation of the telegraphic sentence compression can act as a summary of a whole document, but given the fact that compression for individual sentences is closer to 0.69, this is yet to be demonstrated. And even if that were true, it is unclear whether the cognitive load of reading a sequence of telegraphic sentences would be that much lower than that of reading the original text.
This paper presents some interesting ideas and is well written, but the content is not quite sufficient for publication. In addition to the clarifications and justifications requested above, the authors are encouraged to apply there methods to full lengths documents, which would make for a more substantial contribution.