  Our analysis of large summarization datasets indicates that redundancy is a
very serious problem when summarizing long documents. Yet, redundancy reduction
has not been thoroughly investigated in neural summarization. In this work, we
systematically explore and compare different ways to deal with redundancy when
summarizing long documents. Specifically, we organize the existing methods into
categories based on when and how the redundancy is considered. Then, in the
context of these categories, we propose three additional methods balancing
non-redundancy and importance in a general and flexible way. In a series of
experiments, we show that our proposed methods achieve the state-of-the-art
with respect to ROUGE scores on two scientific paper datasets, Pubmed and
arXiv, while reducing redundancy significantly.

Computation and Language