Document structure is critical for efficient information consumption. However, it is challenging to encode it efficiently into the mod-ern Transformer architecture. In this work, we present HIBRIDS, which injects Hierarchical Biases foR Incorporating Document Structure into the calculation of attention scores. We further present a new task, hierarchical question-summary generation, for summarizing salient content in the source document into a hierarchy of questions and summaries, where each follow-up question inquires about the content of its parent question-summary pair. We also annotate a new dataset with 6 , 153 question-summary hierarchies labeled on long government reports. Experiment results show that our model produces better question-summary hierarchies than comparisons on both hierarchy quality and content coverage, a finding also echoed by human judges. Additionally, our model improves the generation of long-form summaries from lengthy government reports and Wikipedia articles, as measured by ROUGE scores. Document structure facilitates information searching, reading comprehension, and knowledge acquisition by providing an informative overview of the content (Guthrie et al., 1991; Meyer et al., 1980; Taylor and Beach, 1984; Shavelson, 1974; Jonassen, 1988). Specifically, for summarization, its utility is twofold: (1) Source document structures, such as sections and paragraphs, can be instructive for summary generation (Cohan et al., 2018; Celikyilmaz et al., 2018; Zhang et al., 2019); (2) Structures in output summaries, e.g., time-lines (Shahaf et al., 2012; Wang et al., 2015) or aspects (Angelidis and Lapata, 2018), can also ease content understanding. Nonetheless, state-of-the-art abstractive summarization systems, all built on the Transformer architecture (Zhang et al., 2020; Lewis et al., 2020), use attentions to estimate relations between pairwise tokens and largely ignore document structures. While hierarchical encoding has been investigated (Zhang et al., 2019; Balachandran et al., 2021), its need for training large amounts of additional parameters leads to increased memory footprint and thus limits the allowed input length. As for the output, the structure of single document summaries remains largely flat, such as a list of aspects (Meng et al., 2021). We argue that it is imperative to develop systems that can output summaries with rich structures to support knowledge acquisition, which is especially critical for long documents that cover numerous subjects with varying details (Huang et al., 2021; Kryscinski et al., 2021). This work consists of two main objectives: (1) effectively informing summarization models of the source document's structure, and (2) presenting a new summarization task that produces hierarchically organized question-summary pairs to facilitate information consumption. To this end, we propose HIBRIDS (Hierarchical Biases foR Incorporating Document Structure). 1 We design learnable hierarchical biases , as part of the Transformer attention calculation, to adjust attention weights based on tokens' relative positions with regard to the document structure, inspired by the relative position method that modifies attention calculation (Raffel et al., 2020). Concretely, we leverage the natural structure of a document, i.e., section levels, to construct a document structure tree (Figure 2). Each learnable bias corresponds to the relation between a pair of sections, based on the distance between them in the structure tree. Intuitively, hierarchical biases adjust attention weights between tokens based on how conceptually close/distant their corresponding sections are, and they also enable summarizers to capture long-range 1 Our code and newly collected data can be found at https://shuyangcao.github.io/projects/structure_long_summ . Furthermore, we design a new summarization task, hierarchical question-summary generation : Given a document, automatically generate questions and summaries that are organized hierarchically to lay out details for topics at different levels. As shown in Figure 1, each question asks about salient content of the document (to be summarized) and its child questions focus on content in the corresponding summary. This hierarchy not only exposes salient topics and their relations, but also allows readers to quickly identify aspects of interest to focus on. Our task design is inspired by the top-down knowledge learning process: People start by asking broad questions to acquire general knowledge, and then dive into details (Hintikka, 1981; Stede and Schlangen, 2004). Notably, as there is no available dataset with such annotations, we also label a new dataset, GOVREPORT-QS , consisting of 6 , 153 question-summary (QS) hierarchies for summary paragraphs based on 1 , 714 reports from the GOVREPORT dataset (Huang et al., 2021). Each summary paragraph contains 4 . 07 questions with an average QS hierarchy depth of 2 . 26 levels. We first compare HIBRIDS with models that use structure-aware architectures (Rohde et al., 2021) and linear relative positions (Raffel et al., 2020). We conduct experiments on the hierarchical QS generation dataset using two setups: (1) generating a full hierarchy given the first question, and (2) generating follow-up questions given a QS pair. Automatic evaluation shows that our model produces better follow-up questions and summaries than comparisons, while also achieving better or comparable content coverage of full summaries, when compared with a hierarchical model (Ro-hde et al., 2021) that learns 2 M more parameters. In human evaluation, HIBRIDS is considered to build better hierarchies that require fewer manual corrections with more relevant summaries. We further test on the long document summarization task to produce full summaries using GOVREPORT and a newly collected dataset consisting of about 21 k high-quality biographies with summaries from Wikipedia. Again, our system summaries obtain uniformly higher ROUGE scores than comparisons, demonstrating the generalizability of HIBRIDS. Document Structure-aware Summarization. Structural information has long been leveraged for identifying summary-worthy content, including discourse structures labeled by experts (Marcu, 1997) or automatic parsers (Hirao et al., 2013; Durrett et al., 2016; Xu et al., 2020), and topical structures derived from lexical chains (Barzilay and Elhadad, 1999) or probabilistic models (Barzilay and Lee, 2004; Daum III and Marcu, 2006). Natural structures of documents, such as sentences, have been used for pre-training a sentence-level encoder (Zhang et al., 2019) or inducing dependencies among them (Liu et al., 2019) for building extractive summarization systems. Based on separately encoded paragraphs, deep communica-787 tion agents (Celikyilmaz et al., 2018) and inter-paragraph attentions (Liu and Lapata, 2019) are employed to build abstractive summarization models by exchanging information from different paragraphs. Using section structures, Cohan et al. (2018) design a section-level encoder based on the output of a word-level encoder for long document summarization. Nevertheless, multi-level encoders are more expensive since they introduce a significant amount of parameters and add extra padding at multiple levels of model design. By contrast, HIBRIDS effectively informs models of document structure by introducing a novel bias term in attention calculation among tokens, which only introduces a small number of learnable parameters. Long Document Summarization also benefits from the inclusion of document structure information. For example, extractive summarization methods are developed to combine section-level and sentence-level information encoded by multilevel encoders (Xiao and Carenini, 2019) and include longer context via sliding encoding over sections (Cui and Hu, 2021). Recent work on summarizing long documents focuses on designing efficient Transformers with sparse attentions to produce abstractive summaries for long documents in an end-to-end fashion (Beltagy et al., 2020; Zaheer et al., 2020; Huang et al., 2021). However, they all ignore the natural structure of long documents, such as sections and subsections. Based on a simple design, HIBRIDS can be integrated into any efficient Transformer seamlessly for incorporating document structure information. Generating question-answer (QA) pairs has been studied to facilitate information seeking within documents, mainly for producing questions that can be addressed by short phrases (Du and Cardie, 2018; Liu et al., 2020). Prior work mostly focuses on improving QA pair relevance by leveraging additional QA systems (Sachan and Xing, 2018), measuring roundtrip consistency (Alberti et al., 2019), or refining questions iteratively (Qu et al., 2021). Generating a two-level hierarchy of QA pairs from a given paragraph is investigated by Krishna and Iyyer (2019). Our work is different in at least three aspects. First, our goal is to provide a structured summary that focuses on the salient content of the given document, rather than creating questions about any generic information, as done in most QA data construction (Rajpurkar et al., 2016; Choi et al., 2018). Second, our GOVREPORT-QS data 1.1.1 1 1.1 1.2 2 ROOT 0,0 1,-1 2,-2 1,-1 2,0 -1,1 0,0 1,-1 2,0 3,1 -2,2 -1,1 0,0 3,1 4,2 -1,1 -2,0 -3,-1 0,0 3,1 -2,0 -3,-1 -4,-2 -3,-1 0,0 1 1.1 1.1.1 1.2 2  1  1 . concerns richer hierarchies for presenting content in long documents, e.g., 23 . 6% of our hierarchies contain at least three levels . Our parent-child pairs also cover diverse relations, e.g., adding explanations or expanding the topics, beyond asking about specific details as done in Krishna and Iyyer (2019). Third, our questions are designed to be open-ended and grounded in the given document, so our new task is more suitable for summarization models. In this section, we first introduce how relative positions are defined over the document structure tree. Then we present HIBRIDS, which can be included in encoder self-attentions or decoder cross-attentions to adjust the attention scores based on tokens' relative positions. We first construct a document structure tree (Fig-ure 2, left), by leveraging the natural structure of sections and subsections (henceforth sections) in documents, which is available in our experiment data extracted from government reports and Wikipedia articles. We then capture the relative position between pairwise tokens x and y in two different sections, e.g., S x and S y , with two tree-based measures. (1) PathLen ( x, y ) : the length of the shortest path from S x to S y ; (2) LvlDiff ( x, y ) : the level difference from S x to S y . PathLen is 788 designed to be asymmetric to capture content ordering, i.e., its value is positive if S x appears before S y in the document, and vice versa. Examples are displayed in Figure 2. The design of HIBRIDS is based on a lookup table B [  ,  ] : Each item in it corresponds to a learnable hierarchical bias defined by path length and level difference, which is then used to bias the attention calculation for tokens in different sections. Each head maintains its own lookup table B . We first apply HIBRIDS to Transformer encoder self-attention computation, which is called HIBRIDS-ENC . Given the i -th query q i and the matrix K formed by n keys for all input tokens, HIBRIDS adds a bias for each key, with respect to the i -th query, to attention calculation: a ij = softmax( q i KT + b i ) j (1) where the vector b i = [ b i 1 , . . . , b ij , . . . , b in ] contains the bias terms derived from our hierarchical biases as follows: b ij = B [ PathLen ( i, j ) , LvlDiff ( i, j )] (2) where PathLen ( i, j ) and LvlDiff ( i, j ) are the path length and level difference between the sections that tokens i and j belong to. Note that b ij varies among different heads. HIBRIDS-ENC guides tokens to attend to structurally related tokens during encoding. We then apply HIBRIDS to decoder cross-attention calculation, named as HIBRIDS-DEC , to encourage more coherent generation by establishing better alignment with the source document. At the generation step t , the cross-attention weight to the j -th input token adjusted by bias b tj is obtained similarly as in Eq. 1 with the following modification. We calculate b tj as the weighted sum of the hierarchical biases for all input tokens (indexed with l ) to the j -th token. The weight is chosen as the decoder's second last layer's cross-attention score between the t -th generated token and the l th input token, which is shown to better capture word alignment (Garg et al., 2019; Cao and Wang, 2021a). b tj is only applied to the decoder's last layer with the following formulation: b tj = (cid:88) l a crstl  B [ PathLen ( l, j ) , LvlDiff ( l, j )] (3) where a crstl is the decoder's second last layer's cross-attention weight for the generation step t to the l -th input token. HIBRIDSS with Selected Relations. We further consider only keeping salient relations from the tree to reduce the number of parameters to learn, including self (same section), parent-child , ancestor-descendant , sibling , neighboring in text , and within the same top-level section (e.g., 1.1.1 and 1.2 are both in 1). In total, they account for 21 . 6% of all relation occurrences. The modified HIBRIDSS can also be applied to both encoder and decoder. We introduce a new summarization task in this section: Given a document or several sections of a document, we aim to generate question-summary (QS) pairs that are organized hierarchically. As shown in Figure 1, this QS hierarchy lays out details for topics at multiple levels, with each child QS pair expanding the content of its parent. Our task is motivated by how human learns knowledge in a top-down fashion, where general knowledge is acquired first and details and in-depth content are explored later (Hintikka, 1981). This hierarchy proactively highlights the document structure, to further promote content engagement and comprehension (McKeown et al., 2009). We first annotate a new dataset, GOVREPORT-QS, with hierarchical QS pairs, based on articles and corresponding summaries selected from the GOVREPORT dataset (Huang et al., 2021). As these documents and summaries have 9 , 409 and 553 words on average respectively, directly annotating full documents with a QS hierarchy presents a challenge. To address this, we ask annotators to create hierarchical questions for a selected summary paragraph and only allow them to select complete sentences from the summary paragraph as the corresponding answers. Each question created should be fully addressed by its answer and the answer should not contain information irrelevant to the question. For follow-up questions, they are encouraged to ask about specific details or issue questions that can yield summaries that elaborate from their parents. Annotators are also instructed to construct 789 hierarchies of as many levels as possible. Figure 1 demonstrates how hierarchical questions are created and how answer sentences are selected when annotating a report on the development of renewable energy. To cover more documents and avoid collecting shallow hierarchies, each summary paragraph is annotated by one annotator and we select high-quality summary paragraphs for annotation based on heuristic rules, e.g., each paragraph should have at least 3 sentences and 70 words and an adequate level of abstractiveness as measured by normalized density of extractive fragments (Grusky et al., 2018) (with a threshold of < 0 . 15 ). Annotation instructions and details of paragraph selection are in Appendix A. We hired 11 college students who are native English speakers to carry out the annotation tasks in multiple rounds. Feedback was provided to each annotator after each round. A finalization stage was conducted after collecting all annotations, where 4 high-quality annotators were asked to correct typos, remove factoid questions, and make minor adjustment to the hierarchies when errors were detected. GOVREPORT-QS Statistics. In total, 6 , 153 summary paragraphs are annotated with 25 , 055 QS pairs. On average, 4 . 07 QS pairs are created per summary paragraph, spanning 2 . 26 levels. 70 . 5% and 23 . 6% of paragraphs are annotated with two and three levels of questions, making our dataset a valuable benchmark for studying QS hierarchy generation, query-focused summarization, and question generation. The QS hierarchies then become the target generation, and we construct inputs to our QS hierarchy generation system by mapping annotated summary paragraphs back to sections in source documents. Concretely, we match each summary sentence to a document paragraph based on a combination of BERT-based, word overlap-based, and entity overlap-based similarities (details in Appendix A). All sections where matched paragraphs belong, along with the titles of their ancestor sections, are combined together to serve as the system input for generating the corresponding QS hierarchy, as demonstrated in Figure 1. The paired sections have an average length of 2 , 029 , longer than documents in many standard summarization benchmarks. Task I: QSGen-Hier. Based on GOVREPORTQS, we first experiment with a setup where, given the aligned document sections and a root question, the model is expected to produce a summary that addresses the question as well as the rest of the hierarchy. To linearize a QS hierarchy for the Transformer sequential decoder, we concatenate its QS pairs following a depth-first traversal. Special tokens are inserted before each QS pair to indicate the change of its level from the previous QS pair: [L  ] , [L  ] , and [L-] indicate that the level has incremented, decremented, and not changed, respectively. For example, the sample hierarchy in Figure 1 can be formulated as:  A1 [L  ] Q1.1 A1.1 [L-] Q1.2 A1.2 [L  ] Q1.2.1 A1.2.1 . On this task, we divide our samples into train/dev/test splits with sizes of 4 , 878 / 644 / 631 . Task II: QSGen-ChildQ. Next, we leverage GOVREPORT-QS for follow-up question generation: Given a QS pair and the aligned document sections, we aim to generate all child questions. With this setup, two samples can be created from the example in Figure 1. The first one takes as input Q1 A1 and the aligned sections to generate Q1.1 Q1.2 , whereas the other reads in Q1.2 A1.2 and the aligned sections to produce Q1.2.1 . Here we construct train/dev/test splits with sizes of 7 , 157 / 958 / 942 . Task III: Full Summary Generation. We also conduct experiments on GOVREPORT to test HIBRIDS on generating long-form summaries for long inputs. We use the original data splits with 17 , 516 / 974 / 973 samples in train/dev/test sets. We further collect a new dataset from WikiProject Biography 2 (WIKIBIOSUM ) to perform biography summarization. After collecting all available biographies, we keep the ones with at least two levels of section hierarchy and preserve section structures of all levels. For each article, the paragraph before the first section is treated as the target summary, and the rest becomes the input. The finalized dataset has 20 , 833 pairs, divided into 18 , 751 / 1 , 041 / 1 , 041 samples for train/dev/test sets. The average lengths of the input and output for WIKIBIOSUM are 3 , 478 and 1 , 266 . Details of WIKIBIOSUM data collection and filtering procedures are in Appendix B. We set the maximum input length to 5 , 120 for QSGen-Hier, QSGen-ChildQ, and full document summarization on WIKIBIOSUM . On GOVREPORT , the limit is set to 16 , 384 . Evaluation Metrics. We use ROUGE (Lin, 2004) for summarization evaluation and additionally report BLEU up to 4-gram (Papineni et al., 2002) for evaluating the generated questions. We propose to evaluate the generated QS hierarchy against the reference hierarchy with F1 scores calculated as follows, inspired by labeled attachment score in dependency parsing (Zeman et al., 2017): We first map each generated QS pair to a reference QS pair following the highest sum of ROUGE-1 and ROUGE-2 scores between their summaries. After that, we consider two QS pairs with parent-child relation in the generated hierarchy. A match is established only when their mapped QS pairs have a parent-child or ancestor-descendant relation in the reference hierarchy. Precision can then be calculated based on the matching results. We further weight each match based on the sum of the ROUGE-1 and ROUGE-2 scores calculated over both parent and child summaries. Weighted recall and F1 are calculated similarly. Comparisons. All tasks in this work involve long inputs. To allow efficient encoding, we use LONGFORMER (Beltagy et al., 2020) with a window size of 1024 as the base model, and fine-tune it for all systems and comparisons. We first consider comparisons by adding special tokens to encode document structure: (1) SECTOK inserts a special token [SEC] at the start of each section. (2) LVLSECTOK further differentiates sections at varying levels using different tokens (e.g., [SEC-L1] for 1, [SEC-L2] for 1.1). Based on LVLSECTOK , we build all HIBRIDS variants and other comparisons listed below:  HIERENC : We implement the hierarchical model by Rohde et al. (2021), where we replace its sentence encoder with a section encoder of 12 layers to maintain section structures. Among all models, HIERENC requires the most architecture change and adds the most parameters to learn. the selected relations used by HIBRIDSS (3) in a multi-task prediction setup with a bilinear classifier, operating on the representations of section tokens. We use equal weights for prediction loss and summarization loss. TOKBIAS uses linear relative position biases as in T5 (Raffel et al., 2020), which changes Eq. 2 to b ij = R [ i  j ] where R [  ] is a lookup table with each item corresponding to a learnable bias for a given relative distance. SECBIAS replaces token-level linear distance in TOKBIAS with section-level linear distance. Notably, LONGFORMER and models using special tokens have 4 . 59 M parameters. HIBRIDS and models with linear relative position biases use about 4 . 60 M parameters in total. On the other hand, HIERENC and MULTITASK modify the architecture and have 6 . 62 M and 4 . 66 M parameters, which is less efficient for learning compared with models that use bias terms to adjust attention calculation. Results on QSGen-Hier. We report results on the task of generating QS hierarchies in Table 1. HIBRIDS-ENC uniformly outperforms other variants and all comparisons on all metrics, except 791 U.S. Attorney's Office Actions to Enforce the LDA The Office stated that it has sufficient authority and resources to enforce compliance with LDA requirements, including imposing  Q1: What authority does the Office for the District of Columbia have in regard to LDA requirements? for ROUGE-1 and ROUGE-L scores by HIERENC . Note that HIERENC learns 2 M more new parameters than our models, and it produces QS hierarchies of lower quality despite its competitive ROUGE scores (Figure 3). This signifies the effectiveness of our design that directly injects structural information into word-level relation computation. Meanwhile, HIBRIDS on encoder is better at hierarchy quality than its variant on decoder , suggesting the importance of resolving section relations during encoding. Though not reported here, we experiment with HIBRIDS on both the encoder and the decoder, and it results in degraded performance. One possible cause is that HIBRIDS functions differently in these two setups (discussed in 7). We will explore better fusion techniques in future work. Results on QSGen-ChildQ. Results on generating follow-up questions further validate the usefulness of hierarchical biases as shown in Table 2, where questions generated by HIBRIDS -ENC have the best quality as measured by all metrics except for BLEU. SECBIAS , which is aware of section-level linear distance, also obtains outstanding performance, since it focuses on intra-section information and thus better determines what child questions should be asked for better relevance. Human evaluation is conducted on QSGen-Hier, for five models with the highest automatic scores, to help understand how well the generated hierarchies are structured. We hire three judges who Model R1 R2 RL B4 LONGFORMER 26.90 8.69 25.57 14.44 SECTOK 26.76 8.82 25.42 14.51 LVLSECTOK 26.80 8.75 25.52 14.33 Structure-aware Comparisons HIERENC 26.38 8.81 24.99 14.54 MULTITASK 26.84 8.46 25.41 14.59 Models with Linear Bias TOKBIAS 26.73 8.69 25.38 14.43 SECBIAS 27.25 9.07 25.92 14.76 Our Models HIBRIDS-ENC 27.33 9.46 26.00 14.73 HIBRIDSS-ENC 26.41 8.74 24.99 14.44 HIBRIDS-DEC 27.17 8.67 25.71 14.36 HIBRIDSS-DEC 26.29 8.50 25.09 14.30 Table 2: Results for QSGen-ChildQ. have extensive experience in summarization annotation and evaluation tasks to assess 50 groups of question-summary hierarchies. Human inspection on randomly selected outputs shows that most system generations have an appropriate coverage of the salient content in the source. Therefore, we focus on evaluating both global coherence and local coherence of the QS hierarchies based on the following two aspects. First, we ask evaluators to correct each generated hierarchy by rearranging the QS pairs so that each pair is attached to the parent that forms the best follow-up relation in steps. For each step, they are only allowed to attach a pair to its grandparent or sibling (i.e., the parent or child of its current parent). They then report the number of edits conducted for the rearrangement. Second, for each QS pair, we ask them to determine if the question can be answered by the summary. Details of human evaluation are in Appendix C. As can be seen from Table 3, QS hierarchies generated by HIBRIDS-ENC model contain the best structured summaries as they require the fewest number of corrections and the generated questions are also more likely to be addressed by the corresponding summaries. Despite being competitive on automatic metrics, SECTOK generates hierarchies that require the most corrections. Upon additional inspection, we find that HIBRIDS's outputs often have better local coherence than the comparisons. Additionally, all models struggle to generate more engaging questions, which poses another challenge to future studies. As demonstrated in Figure 4, HIBRIDS with full hierarchical biases outperform all comparisons on both datasets, suggesting that our design of including structural relations in bias terms can generalize to other tasks. Compared to the results on QS hierarchy generation, using HIBRIDS on the decoder yields greater improvement on full summary generation, especially in the biography domain where HIBRIDS-DEC obtains the best performance. It is likely that the longer summary length and higher compression ratio on WIKIBIOSUM ( 1 , 266 and 0 . 45 ) makes generation coherence more important by using better alignment. This highlights how hierarchical biases can aid long text generation. Here we aim to understand what is learned by our hierarchical biases. For HIBRIDS-ENC and HIBRIDS-DEC trained on QSGen-Hier, we visualize the values of their learned hierarchical biases averaged over all heads at all layers for each (path length, level difference) pair on an example structure. Additional visualization is in Appendix D. From Figure 5 we see that using HIBRIDS on the encoder encourages models to encode various relations, e.g., by upweighing grandparent (1.1.1 to 1, 1.1.1.1 to 1.1) and preceding sibling (1.2 to 1.1), and downweighing children (1 to 1.1 and 1.2, 1.1 to 1.1.1). This highlights the need of learning heterogeneous relations among sections beyond token distances. By contrast, HIBRIDS on the decoder consistently biases towards parent and sibling contexts. It might be because that the generation of fluent and coherent question-summary pairs relies on being aware of the scope of sections at the same or higher levels. We examine which design choices contribute the most to the performance gain by HIBRIDS, by carrying out ablation studies on QSGen-Hier with HIBRIDS-ENC . We consider taking out (1) level difference, (2) path length, and (3) asymmetry of path length. As shown in Table 4, removing any component reduces summaries' content coverage and hierarchy quality, underscoring their contributions in more precisely representing structural relations for better document encoding. Level difference adds the most to hierarchy quality, as levels 793 Summary Question Hierarchy Model RL B4 F1 HIBRIDS-ENC 38.03 10.16 13.26 w/o Level Difference 0.50 0.08 0.51 w/o Path Length 0.43 +0.05 0.18 w/o Asymmetric Path 0.15 0.12 0.18 Table 4: Ablation study results. We further study if HIBRIDS can boost the section encoder of HIERENC . Table 5 shows that HIERENC with HIBRIDS gains further improvements on generating QS hierarchies and full document summarization on GOVREPORT . This points to promising future adoptions of HIBRIDS by existing models that would benefit from encoding document structure. We present HIBRIDS, which effectively and efficiently injects document structure information into abstractive summarization models via hierarchical learnable biases that adjust the attention score matrix. A new task, hierarchical question-summary generation, is then introduced for generating hierarchically organized question-summary pairs, to expose document structure and salient content to readers. We annotate a new dataset consisting of 6 , 153 summary paragraphs with question-summary hierarchies to facilitate our study, and it can also be used for query-focused summarization and question generation. Experiments on hierarchical question-summary generation and full summary generation show that HIBRIDS produces question-summary hierarchies of higher quality as measured by both automatic metrics and human judges, and achieves higher content coverage of summaries than competitive comparisons as reported by ROUGE. This work is supported in part by National Science Foundation through grant IIS-2046016, Oracle Cloud credits and related resources provided by the Oracle for Research program. We thank the anonymous reviewers for their valuable suggestions. Collection of GOVREPORT-QS and WIKIBIOSUM . We comply with the terms of use and copyright policies of all data sources during the collection of GOVREPORT-QS and WIKIBIOSUM . Personal and other sensitive information is not collected to ensure the privacy of content creators. Before annotating GOVREPORT-QS, we obtain consents from the annotators and inform them of their rights to temporarily suspend or quit the annotation process. During annotation, annotators are fairly compensated (  $15 per hour). Limitations and Potential Risks of HIBRIDS and GOVREPORT-QS. While our experiments focus on datasets consisting of formal long documents, we recognize that long documents could be written in informal languages where our model might not perform reasonably and could generate degraded or even incorrect outputs. Despite recent advancement in improving summary factuality along with its evaluation (Kryscinski et al., 2020; Goyal and Durrett, 2020; Scialom et al., 2021; Cao and Wang, 2021b), the accuracy of existing factuality evaluation metrics has not been verified on long documents, which further increases the risk of incorrect outputs by our model. As our GOVREPORT-QS is based on reports from the United States (US) Government, the topics covered by the dataset are mostly relevant to the national interest of US. Therefore, models trained on our dataset might not be suitable for producing structured summaries for documents published by other countries that focus on other topics. Moreover, our GOVREPORT-QS might bias the model towards a pro-US perspective, which could produce outputs that are harmful to certain populations.
abstain abstain method objective objective result result abstain other abstain abstain abstain abstain objective objective method method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain other other other other other other other other other other other other other other other abstain abstain abstain other abstain method objective abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method method abstain abstain method abstain method abstain abstain method abstain method method method method abstain abstain abstain method abstain abstain method method method abstain abstain method abstain abstain method method method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain objective abstain other other abstain abstain abstain abstain abstain abstain method abstain abstain method method