what state of the art models did they compare with?
HiStGen_P, HiStGen_S, HiStGen_H, HiStGen_R, HiStGen_PSHR, IGCRF+TextRank, IGCRF+TopicRank, IGCRF+Hier, IGCRF+GHD, IGGPD+TextRank, IGGPD+TopicRank, IGGPD+Hier, IGGPD+GHD, GATextRank, GATopicRank, GAHier, GAGHD
In this paper, we introduce and tackle the Outline Generation (OG) task, which aims to unveil the inherent content structure of a multi-paragraph document by identifying its potential sections and generating the corresponding section headings. Without loss of generality, the OG task can be viewed as a novel structured summarization task. To generate a sound outline, an ideal OG model should be able to capture three levels of coherence, namely the coherence between context paragraphs, that between a section and its heading, and that between context headings. The first one is the foundation for section identification, while the latter two are critical for consistent heading generation. In this work, we formulate the OG task as a hierarchical structured prediction problem, i.e., to first predict a sequence of section boundaries and then a sequence of section headings accordingly. We propose a novel hierarchical structured neural generation model, named HiStGen, for the task. Our model attempts to capture the three-level coherence via the following ways. First, we introduce a Markov paragraph dependency mechanism between context paragraphs for section identification. Second, we employ a section-aware attention mechanism to ensure the semantic coherence between a section and its heading. Finally, we leverage a Markov heading dependency mechanism and a review mechanism between context headings to improve the consistency and eliminate duplication between section headings. Besides, we build a novel WIKIOG dataset, a public collection which consists of over 1.75 million document-outline pairs for research on the OG task. Experimental results on our benchmark dataset demonstrate that our model can significantly outperform several state-of-the-art sequential generation models for the OG task.
Document understanding is one of the critical and challenging tasks in information processing. There have been many related research topics in this direction, such as keyword detection BIBREF0 , BIBREF1 , topic modeling BIBREF2 , BIBREF3 , headline generation BIBREF4 , BIBREF5 and text summarization BIBREF6 , BIBREF7 . Keyword detection and topic modeling aim to describe a document by a few important words or topics (i.e., distributions of words) for concise representation; While headline generation and text summarization attempt to compress the document into one or a few sentences to capture the key information. As we can see, most existing research on document understanding has focused on the coarse-grained understanding of documents by capturing its global semantics. In this paper, we attempt to provide fine-grained understanding of documents by unveiling its inhere content structure BIBREF8 , BIBREF9 , i.e., to understand how the document is organized and what it talks about in each part . We thus introduce the Outline Generation (OG) task in this work. Given a multi-paragraph document, the OG task aims to identify its potential sections and generate the corresponding section headings. Figure FIGREF3 shows some typical outline of articles, where Figure FIGREF3 (a) depicts the outline of a Wikipedia article with a two-level hierarchy, and Figure FIGREF3 (b) depicts a typical outline of a research paper. As we can see, the outline can clearly capture the content structure of a document with concise text descriptions (i.e., section headings), which can not only help navigate the reading but also significantly reduce the cognitive burden over the document. Moreover, outlines can also facilitate a variety of text analysis applications such as text clustering and topic survey. In a conceptual level, the OG task could be viewed as a kind of summarization task. However, from the examples shown in Figure FIGREF3 , we can find clear differences between the OG task and traditional summarization tasks. Firstly, the OG task produces a structured output with short descriptions (i.e., keywords or key phrases), while the output of traditional summarization is usually a set of unstructured sentences. Secondly, the OG task needs to summarize the paragraphs (into sections) in a strict sequential order, while the sentences in traditional summarization usually do not map to the paragraphs linearly. Thirdly, the section headings in one outline usually follow a similar style (e.g., topical headings as in Figure FIGREF3 (a) and functional headings as in Figure FIGREF3 (b)), while there is no such requirements in traditional summarization. Therefore, the OG task is actually a novel structured summarization task with its own special challenges. If we take a further look at the OG task, we can find there are actually two structured prediction problem within it, i.e., to identify a sequence of sections (i.e., paragraphs with coherent information/topics), and to generate a sequence of section headings (i.e., short descriptions that summarize the sections) accordingly. Both problems are non-trivial. For section identification, it is unknown how many sections there are in a document. For section heading generation, headings should be able to reflect the section content in a consistent style. To achieve these two goals, an ideal OG model should be able to capture three levels of coherence, namely the coherence between context paragraphs, that between a section and its heading, and that between context headings. The first one is the foundation for section identification, while the latter two are critical for consistent heading generation. In this work, we formulate the OG task as a hierarchical structured prediction problem and introduce a novel hierarchical structured neural generation model, named HiStGen, to solve it. In this model, we view the section boundary prediction problem as a first-level sequential labeling process, and the section heading generation as a second-level structured prediction which depends on the predicted boundary labels from the lower level. For section identification, we employ a Markov paragraph dependency mechanism to model the coherence in adjacent paragraphs to help decide the section boundaries. For section heading generation, we leverage a section-aware attention mechanism BIBREF10 to allow the decoder to focus on the most informative content within a section for heading generation. Furthermore, we introduce a Markov heading dependency mechanism and a review mechanism BIBREF11 between context headings. The Markov heading dependency mechanism is used for modeling the consistency between adjacent headings, while the review mechanism is employed to avoid the repetition in the generated headings. To facilitate the study and evaluation of the OG task, we build a new benchmark dataset based on Wikipedia articles. As we can see, in most multi-paragraph Wikipedia articles, human editors would segment the article into several sections and provide the outline as an overview of the content structure. Therefore, we can directly leverage these articles to build the benchmark. Specifically, we collect Wikipedia articles with outlines under “celebrity", “cities” and “music” category, and obtain hundreds of thousands of articles respectively. We remove the outlines from Wikipedia articles to form the raw text input. The task is to recover the sections and section headings simultaneously. We call this benchmark dataset as WIKIOG. For evaluation, we compare with several state-of-the-art methods to verify the effectiveness of our model. Empirical results demonstrate that outline generation for capturing the inherent content structure is feasible and our proposed method can outperform all the baselines significantly. We also provide detailed analysis on the proposed model, and conduct case studies to provide better understanding on the learned content structure. The main contributions of this paper include: