Traditional Question Generation (TQG) aims to generate a question given an input passage and an answer. When there is a sequence of answers, we can perform Sequential Question Generation (SQG) to produce a series of interconnected questions. Since the frequently occurred information omission and coreference between questions, SQG is rather challenging. Prior works regarded SQG as a dialog generation task and recurrently produced each question. However, they suffered from problems caused by error cascades and could only capture limited context dependencies. To this end, we generate questions in a semi-autoregressive way. Our model divides questions into different groups and generates each group of them in parallel. During this process, it builds two graphs focusing on information from passages, answers respectively and performs dual-graph interaction to get information for generation. Besides, we design an answer-aware attention mechanism and the coarse-to-fine generation scenario. Experiments on our new dataset containing 81.9K questions show that our model substantially outperforms prior works. Question Generation (QG) aims to teach machines to ask human-like questions from a range of inputs such as natural language texts (Du et al., 2017), images (Mostafazadeh et al., 2016) and knowledge bases (Serban et al., 2016). In recent years, QG has received increasing attention due to its wide applications. Asking questions in dialog systems can enhance the interactiveness and persistence of human-machine interactions (Wang et al., 2018). QG bene-fits Question Answering (QA) models through data augmentation (Duan et al., 2017) and joint learning (Sun et al., 2019). It also plays an important role in education (Heilman and Smith, 2010) and clinical (Weizenbaum et al., 1966) systems. Traditional Question Generation (TQG) is de-fined as the reverse task of QA, i.e., a passage and an answer (often a certain span from the passage) are provided as inputs, and the output is a question grounded in the input passage targeting on the given answer. When there is a sequence of answers, we can perform Sequential Question Generation (SQG) to produce a series of interconnected questions. Table 1 shows an example comparing the two tasks. Intuitively, questions in SQG are much more concise and we can regard them with given answers as QA-style conversations. Since it is more natural for human beings to test knowledge or seek information through coherent questions (Reddy et al., 2019), SQG has wide applications, e.g., enabling virtual assistants to ask questions based on previous discussions to get better user experiences. SQG is a challenging task in two aspects. First, information omissions between questions lead to complex context dependencies. Second, there are frequently occurred coreference between questions. Prior works regarded SQG as a dialog generation task (namely conversational QG) where questions are generated autoregressively (recurrently), i.e., a new question is produced based on previous outputs. Although many powerful dialog generation models can be adopted to address the challenges mentioned above, there are two major obstacles. First, these models suffer from problems caused by error cascades. Empirical results from experiments reveal that the later generated questions tend to become shorter with lower quality, especially becoming more irrelevant to given answers, e.g., Why?, What else?. Second, models recurrently generating each question struggle to capture complex context dependencies, e.g., long-distance coreference. Essentially, SQG is rather different from dialog generation since all answers are given in advance and they act as strict semantic constraints during text generation. To deal with these problems, we perform SQG in a semi-autoregressive way. More specifically, we divide target questions into different groups (ques-tions in the same group are closely-related) and generate all groups in parallel. Especially, our scenario becomes non-autoregressive if each group only contains a single question. Since we eliminate the recurrent dependencies between questions in different groups, the generation process is much faster and our model can better deal with the problems caused by error cascades. To get information for the generation process, we perform dual-graph interaction where a passage-info graph and an answer-info graph are constructed and iteratively updated with each other. The passage-info graph is used for better capturing context dependencies, and the answer-info graph is used to make generated questions more relevant to given answers with the help of our answer-aware attention mechanism. Besides, a coarse-to-fine text generation scenario is adopted for the coreference resolution between questions. Prior works performed SQG on CoQA (Reddy et al., 2019), a high-quality dataset for conversational QA. As will be further illustrated, a number of data in CoQA are not suitable for SQG. Some researchers (Gao et al., 2019) directly discarded these data, but the remaining questions may become incoherent, e.g., the antecedent words for many pronouns are unclear. To this end, we build a new dataset from CoQA containing 81.9K relabeled questions. Above all, the main contributions of our work are:  We build a new dataset containing 7.2K passages and 81.9K questions from CoQA. It is the first dataset specially built for SQG as far as we know. We perform semi-autoregressive SQG under dual-graph interaction. This is the first time that SQG is not regarded as a dialog generation task. We also propose an answer-aware attention mechanism and a coarse-to-fine generation scenario for better performance. We use extensive experiments to show that our model outperforms previous work by a substantial margin. Further analysis illustrated the impact of different components. Dataset for this paper is available at https:// github.com/ChaiZ-pku/Sequential-QG . TQG was traditionally tackled by rule-based methods (Lindberg et al., 2013; Mazidi and Nielsen, 2014; Hussein et al., 2014; Labutov et al., 2015), e.g., filling handcrafted templates under certain transformation rules. With the rise of data-driven learning approaches, neural networks (NN) have gradually taken the mainstream. Du et al. (2017) pioneered NN-based QG by adopting the Seq2seq architecture (Sutskever et al., 2014). Many ideas were proposed since then to make it more powerful, including answer position features (Zhou et al., 2017), specialized pointer mechanism (Zhao et al., 2018), self-attention (Scialom et al., 2019), answer separation (Kim et al., 2019), etc. In addition, enhancing the Seq2seq model into more complicated structures using variational inference, adversarial training and reinforcement learning (Yao et al., 2018; Kumar et al., 2019) have also gained much attention. There are also some works performing TQG under certain constraints, e.g., controlling the topic (Hu et al., 2018) and difficulty (Gao et al., 2018) of questions. Besides, combining QG with QA (Wang et al., 2017; Tang et al., 2017; Sun et al., 2019) is also focused by many researchers. As human beings tend to use coherent questions for knowledge testing or information seeking, SQG plays an important role in many applications. Prior works regarded SQG as a dialog generation task (namely conversational QA). Pan et al. (2019) pre-trained a model performing dialog generation, and then fine-tuned its parameters by reinforcement learning to make generated questions relevant to given answers. Gao et al. (2019) iteratively generated questions from previous outputs and leveraged off-the-shelf coreference resolution models to introduce a coreference loss. Besides, additional human annotations were performed on sentences from input passages for conversation flow modeling. Since SQG is essentially different from dialog generation, we discard its dialog view and propose the first semi-autoregressive SQG model. Compared with using the additional human annotation in Gao et al. (2019), our dual-graph interaction deals with context dependencies automatically. Besides, our answer-aware attention mechanism is much simpler than the fine-tuning process in Pan et al. (2019) to make outputs more answer-relevant. As the reverse task of QA, QG is often performed on existing QA datasets, e.g., SQuAD (Rajpurkar et al., 2016), NewsQA (Trischler et al., 2016), etc. However, questions are independent in most QA datasets, making TQG the only choice. In recent years, the appearance of large-scale conversational QA datasets like CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018) makes it possible to train data-driven SQG models, and the CoQA dataset was widely adopted by prior works. Since the test set of CoQA is not released to the public, its training set (7.2K passages with 108.6K questions) was split into new training and validation set, and its validation set (0.5K passages with 8.0K questions) was used as the new test set. responding evidence highlighted in the passage. This brings a big trouble for QG. As an example, consider the yes/no questions counting for 19.8% among all questions. Given the answer  yes  and a corresponding evidence  ...the group first met on July 5 , 1967 on the campus of the Ohio state uni-versity... , there are many potential outputs, e.g.,  Did the group first met in July? ,  Was the group first met in Ohio state? . When considering the context formed by previous questions, the potential outputs become even more (the original question in CoQA is  Was it founded the same year? ). When there are too many potential outputs with significantly different semantic meanings, training a converged QG model becomes extremely difficult. For this reason, Gao et al. (2019) directly discarded questions that cannot be answered by spans from passages. However, the remaining questions can become incoherent, e.g., antecedent words for many pronouns become unclear. To this end, we build a new dataset from CoQA by preserving all 7.7K passages and rewriting all questions and answers. More specifically, we first discarded questions that are unsuitable for SQG. To do so, three annotators were hired to vote for the preservation/deletion of each question. A question is preserved if and only if it can be answered by a certain span from the input passage 2 . As a result, most deleted questions were yes/no questions and unanswerable questions. Besides, the kappa score between results given by different annotators was 0.83, indicating that there was a strong interagreement between annotators. For the remaining QA-pairs, we preserved their original order and replaced all answers by spans from input passages. After that, we rewrote all questions to make them coherent. To avoid over-editing, annotators were asked to modify as little as possible. It turned out that in most cases, they only needed to deal with coreference since the prototype of pronouns were no longer existed. To further guarantee the annotation quality, we hired another project manager who daily examined 10% of the annotations from each annotator and provided feedbacks. The annotation was considered valid only when the accuracy of examined results surpasses 95%. Our annotation process took 2 months, and we finally got a dataset containing 7.7K passage with 81.9K QA-pairs. 2 Using certain spans from input passages (instead of free-formed text) as answers is a conversion in QG. In this way, the number of potential output questions is greatly reduced. As shown in Figure 1, the model first builds a passage-info graph and an answer-info graph by its passage-info encoder and answer-info encoder respectively. After that, it performs dual-graph interaction to get representations for the decoder. Finally, different groups of questions are generated in parallel under a coarse-to-fine scenario. Both encoders and decoder take the form of Transformer architecture (Vaswani et al., 2017). In SQG, we input a passage composed by n sentences P = { S i } ni =1 and a sequence of l answers { A i } li =1 , each A i is a certain span of P . The target output is a series of questions { Q i } li =1 , where Q i can be answered by A i according to the input passage P and previous QA-pairs. As mentioned above, we perform SQG in an semi-autoregressive way, i.e., target questions are divided into into different groups. Ideally, questions in the same group are expected to be closely-related, while questions in different groups should be as independent as possible. Our model takes a simple but effective unsupervised question clustering method. The intuition is: if two answers come from the same sentence, the two corresponding questions are likely to be closely-related. More specifically, if the k -th sentence S k contains p answers from { A i } li =1 , we cluster them into an answer-group G ansk = { A j 1 , A j 2 , ..., A j p } where j 1 < j 2 < ... < j p are continuous indexes from { 1 , 2 , ..., l } . By replacing each answer in G ansk with its corresponding question, we get a question-group G quesk = { Q j 1 , Q j 2 , ..., Q j p } , and we further define a corresponding target-output T k as  Q j 1 [ sep ] Q j 2 [ sep ] ... [ sep ] Q j p  where  [ sep ]  is a special token. In Table 1, there are four target outputs T 1 , T 2 , T 4 , T 5 (no T 3 since the third sentence in Table 1 do not contain any answer), T 2 is  What was he doing there? [sep] On What? [sep] ... [sep] What was Tim doing?  corresponding with the second sentence, and T 5 is  What did he say?  corresponding with the last sentence. Supposing there are m answerand question-groups, then our model generates all the m target-outputs in parallel, i.e., all questions are generated in a semi-autoregressive way. As shown in Figure 1, our passage-info encoder maps input sentences { S i } n i =1 into their sentence representations { s i } ni =1 where every s i  R 2 d s . We regard each sentence as a sequence of words and replace each word by its pre-trained word embeddings (Mikolov et al., 2013) which is a dense vector. After that, the sequence of word embeddings is sent to a Transformer-encoder that outputs a corresponding sequence of vectors. By averaging these vectors, we get the local representation s locali  R d s of S i . After we get the local representations of all sentences { S i } ni =1 in passage P , another Transformer-encoder is adopted to map the sequence { s locali } ni =1 into { s globali } ni =1 , where s globali  R d s is called the Figure 2: Illustration of answer embeddings and an answer-attention head for the forth sentence in Table In other words, the passage-info encoder takes a hiarachical structure. We expect the local and global representations capture intraand intersentence context dependencies respectively, and the final representation for S i is s i = [ s locali ; s globali ]  R 2 d s . As described in Section 4.1, the input answers are split into m answer-groups. For G ansk corresponding with the k -th sentence of the input passage, we define { G ansk , S k } as a rationale R k , and further obtain its representation r k  R 2 d r by our answer-info encoder, which is based on a Transformer-encoder regarding sentence S k as its input. To further consider information from G ansk , two more components are added into the answer-info encoder, as shown in Figure 2. First, we adopt the answer-tag features. For each word w i in sentence S k , the embedding layer computes [ x w i ; x a i ]  R d r as its final embedding, where x w i is the pre-trained word embedding and x a i contains answer-tag features. More specifically, we give w i a label from { O, B, I } if it is outside, the beginning of, inside of any answer from G ansk , and use a vector corresponding with this label as x a i . Second, we design the answer-aware attention mechanism. In the multi-head attention layer, there are not only l h vanilla self-attention heads, but also l a answer-aware heads for each answer in G ansk . In an answer-aware head corresponding with answer A , words not belonging to A are masked out during the attention mechanism. The output of the Transformer-encoder is a sequence of vectors H enck = { h enck } ( h enck  R d r ) corresponding with the input word sequence from S k . In our SQG task, the input passage contain n sentences, which can be represented by { s i } ni =1  R 2 d s leveraging the passage-info encoder. Among all input sentences, only m of them contain certain answers ( m  n ), and we further define m rationales based on these sentences, { G ansF ( j ) , SF ( j ) } mj =1 , where the j -th rationale ( j  { 1 , 2 , ..., m } ) corresponds with the F ( j ) -th sentence of the input passage ( F ( j )  { 1 , 2 , ..., n } ). For the example in Table 1, n = 5 , m = 4 , F ( j ) maps { 1 , 2 , 3 , 4 } into { 1 , 2 , 4 , 5 } respectively. Using the answer-info encoder, we can get representations { r F ( j ) } mj =1  R 2 d s for all rationales. We further build a passage-info graph V and an answer-info graph U based on these representations. For the rationale corresponding with the k -th sentence of the input passage, we add node u k , v k in graph U , V respectively. For the example in Table 1, U is compused by { u 1 , u 2 , u 4 , u 5 } and V is compused by { v 1 , v 2 , v 4 , v 5 } , as shown in Figure 1. The initial representation for u k is computed by: u (0) k = ReLU ( W u [ r k ; e k ] + b u )  R d g (1) where r k  R 2 d r is the rationale representation, e k  R d e is the embedding of index k , and W u  R ( d e +2 d r )  d g , b u  R d g are trainable parameters. And the initial representation for v k is: v (0) k = ReLU ( W v [ s k ; e k ] + b v )  R d g (2) where s k  R 2 d s is the sentence representation and W v  R ( d e +2 d s )  d g , b v  R d g are parameters. After adding these points, there are m nodes in U and V respectively. For u i , u j  U corresponding with the i -th, j -th input sentences respectively, we add an edge between them if | i  j | <  (  is a hyper-parameter). Similarly, we add edges into V and the two graphs are isomorphic. In our answer-info graph U , node representations contain information focused on input answers. In the passage-info graph V , node representations capture interand intra-sentence context dependencies. As mentioned above, a good question should be answer-relevant as well as capturing complex context dependencies. So we should combine information in both U and V . Our dual-graph interaction is a process where U and V iteratively update node representations with each other. At time step t , representations u ( t  1) i , v ( t  1) i are updated into u ( t ) i , v ( t ) i respectively under three steps. First, we introduce the information transfer step . Taking U as an example. Each u ( t  1) i receives a ( t ) i from its neighbors (two nodes are neighbors if there is an edge between them) by: a ( t ) i = (cid:88) u j N ( u i ) W ij u ( t  1) j + b ij (3) where N ( u i ) is composed by all neighbors of node u i and W ij  R d g  d g , b ij  R d g are parameters controlling the information transfer. For u i , u j and u i (cid:48) , u j (cid:48) whose | i  j | = | i (cid:48)  j (cid:48) | , we use the same W and b . In other words, we can first create a sequence of matrices { W 1 , W 2 , ... }  R d g  d g and vectors { b 1 , b 2 , ... }  R d g , and then use | i  j | as the index to retrieve the corresponding W ij , b ij . For graph V , we similarly compute  a ( t ) i = (cid:88) v j N ( v i )  W ij v ( t  1) j +  b ij (4) In the second step, we compute multiple gates . For each u ( t  1) i in U , we compute an update gate y ( t ) i and a reset gate z ( t ) i by: y ( t ) i =  ( W y [ a ( t ) i ; u ( t  1) i ]) z ( t ) i =  ( W z [ a ( t ) i ; u ( t  1) i ]) (5) where W y , W z  R 2 d g  d g are paramenters. Similarly, for each v ( t  1) i in V we compute:  y ( t ) i =  (  W y [ a ( t ) i ; v ( t  1) i ])  z ( t ) i =  (  W z [ a ( t ) i ; v ( t  1) i ]) (6) Finally, we perform the information interaction , where each graph updates its node representations under the control of gates computed by the other graph . More specifically, node representations are updated by: u ( t ) i =  z ( t ) i (cid:12) u ( t  1) i + ( 1   z ( t ) i ) (cid:12) tanh ( W a [ a ( t ) i ;  y ( t ) i (cid:12) u ( t  1) i ]) v ( t ) i = z ( t ) i (cid:12) v ( t  1) i + ( 1  z ( t ) i ) (cid:12) tanh (  W a [  a ( t ) i ; y ( t ) i (cid:12) v ( t  1) i ]) (7) The idea of using gates computed by the other graph to update node representations in each graph enables the information in input passage and answers interact more frequently, both of which act as strong constraints to the output questions. By iteratively performing the three steps for T times, we get the final representations u ( T ) i and v ( T ) i for u i  U and v i  V . For the k -th input sentence S k containing certain answers, our decoder generates the corresponding target-output T k . As mentioned above, the generation process of all target-outputs are independent. The decoder is based on the Transformer-decoder containing a (masked) multi-head self-attention layer, a multi-head encoder-attention layer, a feed-forward projection layer and the softmax layer. To compute keys and values for the multi-head encoder-attention layer, it leverages the outputs from our answer-info encoder, i.e., it uses H enck described in Section 4.3 to generate T k corresponding with the k -th sentence. To generate coherent questions, we need to capture the context dependencies between input answers and passages. To this end, both u ( T ) k and v ( T ) k , which comes from the dual-graph interaction process, are used as additional inputs for generating T k . First, they are concatenated with the output of each head from both (masked) multi-head self-attention layer and multi-head encoder-attention layer before sending to the next layer. Second, they are concatenated with inputs of the feed-forward projection layer. The two representations are also expected to make generated questions more relevant to given inputs. Since the semi-autoregressive generation scenario makes it more challenging to deal with coreferences between questions (especially questions in different groups), we perform question generation in a coarse-to-fine manner. The decoder only needs to generate coarse questions where all pronouns are replaced by a placeholder  [p] . To get final results, we use an additional pre-trained coreference resolution model to fill pronouns into different placeholders. To make a fair comparison, we use the coreference resolution model (Clark and Manning, 2016) adopted by prior works CoreNQG (Du and Cardie, 2018) and CorefNet (Gao et al., 2019). In this section, we first introduce the three kinds of baselines. After that, we compare and analyse the results of different models under both automatic and human evaluation metrics. We compared our model with seven baselines that can be divided into three groups. First, we used three TQG models: the Seq2seq (Du et al., 2017) model which pioneered NN-based QG, the CopyNet (See et al., 2017) model that introduced pointer mechanism, and CoreNQG (Du and Cardie, 2018) which used hybrid features (word, answer and coreference embeddings) for encoder and adopted copy mechanism for decoder. Second, since prior works regarded SQG as a conversation generation task, we directly used two powerful multi-turn dialog systems: the latent variable hierarchical recurrent encoder-decoder architecture VHRED (Serban et al., 2017) , and the hierarchical recurrent attention architecture HRAN (Xing et al., 2018) . Third, we used prior works mentioned above. For Pan et al. (2019), we adopted the ReDR model which had the best performance. For Gao et al. (2019), we used the CorefNet model. Although a CFNet in this paper got better results, it required additional human annotations denoting the relationship between input sentences and target questions. So it is unfair to compare CFNet with other methods. It is worth mentioning that when generating questions using the second and third groups of baselines, only previously generated outputs were used as dialog history, i.e., the gold standard questions are remain unknown (in some prior works, they were directly used as dialog history, which we think is inappropriate in practice). Following the conventions, we used BLEU (Pa-pineni et al., 2002), ROUGE-L (Lin, 2004) and METEOR (Lavie and Agarwal, 2007) as automatic evaluation metrics. We also computed the average word-number of generated questions. As shown in Table 2, our semi-autoregressive model outperformed other methods substantially. When we focus on the second and third groups of baselines regarding SQG as multi-turn dialog generation tasks, we can find that models from the third group are more powerful since they make better use of information from input passages. Besides, models from the second group tend to generate shortest questions. Finally, similar to the problem that dialog systems often generate dull and responses, these models also suffer from producing general but meaningless questions like What?, How?, And else?. When we compare the first and third groups of baselines (which are all QG models), it is not surprising that SQG models show more advantages than TQG models, as they take the relationships between questions into consideration. Besides, CorefNet gets better performance among all baselines, especially ReDR. This indicates that comparing with implicitly performing reinforcement learning through QA models, explicitly using target answers as inputs can be more effective. Note that if we directly compare the performance between SQG task and TQG task under the same model (e.g., the Seq2seq model), evaluation scores for TQG tasks are much higher, which is not surprising since SQG is harder than TQG dealing with dependencies between questions. Another fact lies in the computation of automatic evaluation metrics. As shown in Table 2, questions in SQG datasets are much shorter than TQG. Since our automatic evaluation metrics are based on n -gram overlaps between generated and gold standard questions, the scores significantly go down with the growth of n (for this reason, the BLEU 4 scores are not listed in Table 2). This also illustrates the importance of performing human evaluation. It is generally acknowledged that automatic evaluation metrics are far from enough for SQG. So we perform human evaluation in five aspects. Fluency measures if a question is grammatically correct and is fluent to read. Coherence measures if a question is coherent with previous ones. Coreference measures if a question uses correct pronouns. Answerability measures if a question is targeting on the given answer. Relevance measures if a question is grounded in the given passage. Since performing human evaluation is rather expensive and time-consuming, we picked up the best TQG model (CoreNQG), SQG model (CorefNet) to compare with our model. We randomly selected 20 passages from the test set with 207 given answers and asked 10 native speakers to evaluate the outputs of each model independently. Under each aspect, reviewers are asked to choose a score from { 1, 2, 3 } , where 3 indicates the best quality. The average scores for each evaluation metric are shown in Table 4. We can find that our model gets the best or competitive performance in each metric. When it comes to fluency, all models get high performance, and the CorefNet that outputs BLEU 3 ROUGE METEOR No interact 11.35 37.31 17.05 Uni-graph 9.86 36.44 15.87 Uni-heads 10.33 37.48 16.24 No co2fine 11.75 37.92 17.17 Non-auto 7.79 33.62 14.83 Ours 12.06 38.15 17.26 Table 5: Results for ablation tests. shortest questions gets the best score. As for coherence, CoreNQG gets poor results since it generates questions independently. When it comes to coreference, our model only slightly lower than CorefNet, which added direct supervision to attention weights by a coreference resolution model. Finally, our model gets the best performance on both answer-abity and relevance. However, it is worth noticing that all models get rather poor performances under these two aspects, indicating that making a concise question meaningful (i.e., targeting on given answers) with more information from input passage (i.e., performing proper information elimination) is a major challenge in SQG. Besides, as pointed out by Table 3, questions in our SQG dataset are significantly shorter compared with TQG dataset, making subtle errors much easier to be noticed. In this section, we perform ablation test to verify the influence of different components in our model. First, we modify Equation 7 into u ( t ) i = z ( t ) i (cid:12) u ( t  1) i + ( 1  z ( t ) i ) (cid:12) tanh ( W a [ a ( t ) i ; y ( t ) i (cid:12) u ( t  1) i ]) v ( t ) i = z ( t ) i (cid:12) v ( t  1) i + ( 1   z i ( t ) ) (cid:12) tanh (  W a [  a ( t ) i ;  y ( t ) i (cid:12) v ( t  1) i ]) (8) to get the no interact model, i.e., two graphs are independently updated without any interaction. Second, we build a uni-graph model by removing the passage-info encoder (the remaining rationale graph is updated similarly to Li et al. (2015)). Third, we discard the attention-aware heads in the rationale encoder to get a uni-heads model. Then, we build the no co2fine model without the coarse-to-fine generation scenario. Finally, we build a non-auto model that performs SQG in an non-autoregressive way, i.e., each question is generated in parallel. Peter was a very sad puppy. He had been inside of the pet store for a very long time. In fact, he had been there for [three months] 1 ! Peter had seen many other puppies find a person; he began to wonder why he could not get one. He thought that [maybe his fur was not pretty enough or maybe his bark was not loud enough] 2 . He tried and tried to please every person who came to the store, but they all picked smaller puppies. However, one day all of this changed. [Sammie] 3 came into the store looking for [a golden puppy] 4 . She wanted a puppy she could snuggle with. It so happened that Peter was very sad and tired that day. Sammie came to hold him. Peter wanted to show off [his bark] 5 , but he was [too tired] 6 . He [fell right to sleep] 7 . Sammie loved him at once and loved holding him in her arms. Sammie took [Peter] 8 home that day, and they made lots of fun memories. As shown in Table 5, each component in our model plays an important part. Results for the no interact model indicate that compared with independently updating the passage-info graph and answer-info graph, making these information more interacted by our dual-graph interaction scenario is more powerful. Not surprisingly, the uni-graph model removing the passage encoder (i.e., less focusing on context dependencies between sentences from input passage), and the uni-heads model discarding our answer-aware attention mechanism (i.e., less focusing on given answers) get significant worse performance compared with our full model. Besides, our coarse-to-fine scenario helps to better deal with the dependencies between questions since there are widespread coreferences. Finally, although the architecture of non-auto model is a special case of our model where each group only contains a single question, the performance drops significantly, indicating the importance of using semi-autoregressive generation. However, the dual-graph interaction still makes its performance better than the Seq2seq and CopyNet in Table 2. 6.2 Running Examples In Table 6, we present some generated examples comparing our model and the strongest baseline CorefNet. On the one hand, our model performs better than CorefNet, especially that the output questions are more targeting on given answers (turn 2, 6, 7). It also correctly deals with coreferences (e.g., distinguishing Peter and Sammie). On the other hand, the generated questions have poor quality when gold standard questions involve more reasoning (turn 2, 6). Besides, the gold standard questions are more concise as well (turn 4, 6). In this paper, we focus on SQG which is an important yet challenging task. Different from prior works regarding SQG as a dialog generation task, we propose the first semi-autoregressive SQG model, which divides questions into different groups and further generates each group of closely-related questions in parallel. During this process, we first build a passage-info graph, an answer-info graph, and then perform dual-graph interaction to get representations capturing the context dependencies between passages and questions. These representations are further used during our coarse-to-fine generation process. To perform experiments, we analyze the limitation of existing datasets and create the first dataset specially used for SQG containing 81.9K questions. Experimental results show that our model outperforms previous works by a substantial margin. For future works, the major challenge is generating more meaningful, informative but concise questions. Besides, more powerful question clustering and coarse-to-fine generation scenarios are also worth exploration. Finally, performing SQG on other types of inputs, e.g., images and knowledge graphs, is an interesting topic. This work was supported by National Natural Science Foundation of China (61772036) and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology). We thank the anonymous reviewers for their helpful comments. Xiaojun Wan is the corresponding author.
abstain method abstain abstain abstain method method abstain method objective abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method method abstain abstain abstain abstain objective objective objective method objective objective result objective other other other other other other other other other other other other other objective method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method other method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain method method method abstain other abstain abstain method method method abstain abstain method method abstain objective abstain other method objective method method method other method method abstain abstain method method abstain abstain abstain abstain method abstain abstain method abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain other method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain abstain abstain method objective objective method objective result abstain abstain abstain other other other