  In a world of proliferating data, the ability to rapidly summarize text is
growing in importance. Automatic summarization of text can be thought of as a
sequence to sequence problem. Another area of natural language processing that
solves a sequence to sequence problem is machine translation, which is rapidly
evolving due to the development of attention-based encoder-decoder networks.
This work applies these modern techniques to abstractive summarization. We
perform analysis on various attention mechanisms for summarization with the
goal of developing an approach and architecture aimed at improving the state of
the art. In particular, we modify and optimize a translation model with
self-attention for generating abstractive sentence summaries. The effectiveness
of this base model along with attention variants is compared and analyzed in
the context of standardized evaluation sets and test metrics. However, we show
that these metrics are limited in their ability to effectively score
abstractive summaries, and propose a new approach based on the intuition that
an abstractive model requires an abstractive evaluation.

Computation and Language