Question answering (QA) is a fundamental means to facilitate assessment and training of narrative comprehension skills for both machines and young children, yet there is scarcity of high-quality QA datasets carefully designed to serve this purpose. In particular, existing datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. Drawing on the reading education research, we introduce F airy tale QA 1 , a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Generated by educational experts based on an evidence-based theoretical framework, F airytale QA consists of 10,580 explicit and implicit questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. Our dataset is valuable in two folds: First, we ran existing QA models on our dataset and confirmed that this annotation helps assess models' fine-grained learning skills. Second, the dataset supports question generation (QG) task in the education domain. Through benchmarking with QG models, we show that the QG model trained on F airytale QA is capable of asking high-quality and more diverse questions. Equal contributions ying.xu@uci.edu, dakuo.wang@ibm.com, moyumyu@tencent.com, drritchi@uci.edu, yaob@rpi.edu. Work done while Mo was at IBM . Corresponding Author. 1 Our dataset is available at https://github.com/ uci-soe/FairytaleQAData . [Sect 1] Once upon a time there was a lad who was better o ff than all the others. He was never short of money, for he had a purse which was never empty. ... ... [Sect 6] When the king's daughter had eaten of the apples, she had a pair of horns. But one day a foreign doctor from afar came to court. He was not from their country, he said, and had made the journey purposely just to try his luck here. But he must see the king's daughter alone, said he, and permission was granted him. ... [Sect 8] ...  Q1 :Who will the foreign doctor turn out to be? Reading comprehension is a complex, multidimensional cognitive process (Kim, 2017). Question answering (QA) is fundamental for supporting humans' development of reading comprehen-447 sion skills, as questions serve as both instruments for evaluation and tools to facilitate learning. To achieve this goal, comprehension questions should be valid and reliable, meaning that all items are designed to cohesively assess comprehension rather than some other skills (e.g., text matching, paraphrasing, or memorization) (Roberts and Priest, 2006). Moreover, from the educational perspective, given that reading comprehension is a multicomponent skill, it is ideal for comprehension questions to be able to identify students' performance in specific sub-skills, thus allowing teachers to provide tailored guidance (Francis et al., 2005). However, creating a large and suitable set of questions for supporting narrative comprehension is both time-consuming and cognitively demanding. Some researchers have proposed developing models to automatically generate questions or QA-pairs that satisfy the need for a continuous supply of new questions (Kurdi et al., 2020; Yao et al., 2022), which can potentially enable large-scale development of AI-supported interactive platforms for the learning and assessment of reading comprehension skills (e.g., (Zhang et al., 2022)). However, existing datasets are not particularly suitable for training question generation (QG) models for educational purposes (Das et al., 2021). This is primarily because the datasets are not typically structured around the specific dimensions of reading comprehension sub-skills, nor do they provide su ffi cient information on what sub-skills are tested. Consequently, QG models built on these datasets only yield one single comprehension score without a more detailed breakdown of performance on comprehension sub-skills. This issue is compounded by the fact that many benchmarks rely on crowd-sourced workers who may not have su ffi cient training or education domain knowledge needed to create valid questions in a consistent way. To bridge the gap, we constructed F airytale QA, an open-source dataset focusing on comprehension of narratives, targeting students from kindergarten to eighth grade. We focus on narrative comprehension for two reasons. First, narrative comprehension is a high-level comprehension skill strongly predictive of reading achievement (Lynch et al., 2008) and plays a central role in daily life as people frequently encounter narratives in di ff erent forms (Goldie, 2003). Second, narrative stories have a clear structure of specific elements and relations among these elements, and there are existing validated narrative comprehension frameworks around this structure, which provides a basis for developing the annotation schema for our dataset. We employed education experts who generated 10,580 question-answer pairs based on a collection of 278 fairytale stories for young readers, following evidence-based narrative comprehension frameworks (Paris and Paris, 2003; Alonzo et al., 2009). Thereby, F airytale QA contains questions that focus on several narrative elements and relations, increasing the validity and reliability of the assessment. In addition, F airytale QA also contains both explicit questions that involve answers found explicitly in the text and implicit questions that require high-level summarization (Table 1), thus representing a relatively balanced assessment with questions of varying di ffi culty (Zucker et al., 2010; Raphael, 1986). Most importantly, our selection of annotators with education domain knowledge as well as the training and quality control process ensured that the aforementioned annotation protocol was consistently implemented. A subset of questions in our dataset has been validated with 120 pre-kindergarten and kindergarten students (IRB approved from the first author's institution), proving the questions' reliability and validity. We show the utility of F airytale QA through two benchmarking experiments. First, we used our data to train and evaluate state-of-the-art (SOTA) QA models and demonstrated that (1) F airytale QA contains challenging phenomena for existing models, and (2) it can support finer-grained analysis on the di ff erent types of comprehension sub-skills, even for models trained on general QA datasets (NarrativeQA (Kocisk`y et al., 2018)). We further calibrated model performances with human baseline, highlighting the most visible gap in models' reasoning capabilities on recognizing casual relationships and predicting event outcomes. Second, we used F airytale QA to power question generation and showed that the QG model trained on ours was more capable of asking diverse questions and generating questions with higher quality. Despite a large number of datasets on reading comprehension, few focus on comprehension of narrative text. Table 2 reviews di ff erent narrative-related properties of existing popular QA datasets comparing with our proposed F airytale QA dataset. Narra-448 Dataset Educ. tiveQA (Kocisk`y et al., 2018) is one of the representative datasets. It was generated by crowd workers who wrote QA pairs according to summaries of books or movie scripts, while the task takers are supposed to answer these questions based on their reading of original books or movie scripts. As such, this dataset is posited to evaluate a per-son's understanding of the underlying narrative, with a significant amount of event-related questions (Mou et al., 2021). However, NarrativeQA simply instructed crowd-sourced workers to generate questions as if they were to test students without using a detailed annotation protocol. It is questionable whether these workers actually had experiences in testing students, and the lack of protocol may have imposed too little control over the coverage of reading sub-skills. BookTest (Bajgar et al., 2016) is an automatically constructed cloze-style QA dataset based on a collection of narrative texts retrieved from Project Gutenberg. The questions were generated by automatically removing a noun or entity in a sentence that has appeared in the preceding context. While cloze-style tests can be a valid instrument for assessing reading comprehension, their validity depends on the careful selection of words to be removed so that filling them in requires proper comprehension (Gellert and Elbro, 2013). It is unlikely that automatically constructed cloze tests would meet such standards. Another dataset, TellMeWhy (Lal et al., 2021), aims to facilitate and assess understanding of causal relationships . This dataset contains why questions that are relatively challenging, given that they require additional information not directly provided in the text. However, TellMeWhy only addresses one narrative component type (i.e., causal relationship), whereas F airytale QA provides seven evaluation components. Moreover, TellMeWhy was built upon ROCStories (Mostafazadeh et al., 2016) and thus only examine comprehension on incomplete story exerpts, which may have limited the dataset's ability to assess macro-level summarization and inference making. There are several benchmarks derived from sources for education purposes (e.g., exams or curricula). RACE (Lai et al., 2017) is a large-scale dataset consisting of comprehension questions from English exams for Chinese middle and high school students. RACE uses a mixture of narrative and informational paragraphs. These two genres require slightly di ff erent comprehension skills (Liebfre-und, 2021), and students perform di ff erently based on what genres of text they read (Denton et al., 2015). Mixing these two together in one dataset without annotating the specific genre of each story / question obscures the ability to o ff er a precise assessment. Moreover, RACE is in multiple-choice format, and paragraphs are usually shorter. These two characteristics may make the RACE dataset less challenging, and recent models have demonstrated close-to-human performance 2 . CLOTH (Xie et al., 2017) is a cloze-style dataset also collected from English exams with multiple choice fill-in-the-blank questions. CLOTH can be advantageous for educational QG as each question is labeled with the level of reasoning it involves. However, this dataset shares certain limitations inherent to multiple-choice formats (Klufa, 2015). There are some datasets that are designed for assessing narrative comprehension skills but do not 2 http://www.qizhexie.com/data/RACE_ leaderboard.html use QA as a form of evaluation. Several datasets, such as NovelChapters (Ladhak et al., 2020) and BookSum (Kryscinski et al., 2021), evaluate models' comprehension through summarization tasks. However, there have been debates of whether comprehension can be assessed solely through summarization (Head et al., 1989), as summarization poses a high demand on writing that confounds the reading skills intended to be assessed. Two other recent datasets focus on singular specific elements in narratives. The LiSCU dataset (Brahman et al., 2021) targets readers' understanding of characters , and Sims et al. (2019) propose a dataset for detecting events in narratives. Given their focus on single narrative elements, these two datasets may not provide a comprehensive evaluation of narrative comprehension. We developed the F airytale QA dataset to address some of the limitations in existing benchmarks. Our dataset contains 10,580 QA pairs from 278 classic fairytale stories. In the remainder of this section, we report the dataset construction process and its key statistics. The narrative texts utilized in the dataset are classic fairytales with clear narrative structures. We gathered the text from the Project Gutenberg website 3 , using  fairytale  as the search term. Due to a large number of fairytales found, we used the most popular stories based on the number of downloads since these stories are presumably of higher quality. To ensure the readability of the text, we made a small number of minor revisions to some obviously outdated vocabulary (e.g., changing ere to be-fore) and the unconventional use of punctuation (e.g., changing consecutive semi-colons to periods). For each story, we evaluated the reading di ffi culty level using the textstat 4 Python package, primarily based on sentence length, word length, and commonness of words. We excluded stories that are at 10th grade level or above. These texts were broken down into small sections based on their semantic content by our annotators. The annotators were instructed to split the story into sections of 100-300 words that also contain meaningful content and are separated at 3 https://www.gutenberg.org/ 4 https://pypi.org/project/textstat/ natural story breaks. An initial annotator would split the story, and this would be reviewed by a cross-checking annotator. Most of the resulting sections were one natural paragraph of the original text. However, sometimes several paragraphs were combined (usually dialogue); and some exceptionally long paragraphs that contained more than one focal event were divided into multiple sections. On average, there are 15 sections per story, and each section has an average of 150 words (Table 4). Categorization via Narrative Elements or Relations F airytale QA is intended to include QA pairs that capture the seven narrative elements / relations that are verified in prior educational research (Paris and Paris, 2003). Definitions of question types are shown below. Example questions for each type are in Appendix D.  Character questions ask test takers to identify the character of the story or describe characteristics of characters. Setting questions ask about a place or time where / when story events take place and typically start with  Where  or  When . Action questions ask about characters' behaviors or information about that behavior. Feeling questions ask about the character's emotional status or reaction to certain events and are typically worded as  How did / does / do . . . feel . Causal relationship questions focus on two events that are causally related where the prior events causally lead to the latter event in the question. This type of questions usually begins with  Why  or  What made / makes . Outcome resolution questions ask for identifying outcome events that are causally led to by the prior event in the question. This type of questions are usually worded as  What happened / happens / has happened...after... . Prediction questions ask for the unknown outcome of a focal event, which is predictable based on the existing information in the text. These labels are to ensure the presence of the variety of questions' sub-skills so that the models trained on this dataset can also generate the variety. The labels are not intended to aid the training of a model to classify questions. Some  but not all  of the labels may be determined by surface features. For example, feeling questions typically contain the words feel or feels, while action questions 450 F airytale QA Dataset Train Validation Test 232 Books with 8548 QA-pairs 23 Books with 1025 QA-pairs 23 Books with 1007 QA-pairs Mean S.D. Min Max Mean S.D. Min Max Mean S.D. Min Max # section per story 14.4 8.8 2 60 16.5 10.0 4 43 15.8 10.8 2 55 # tokens per story 2160.9 1375.9 228 7577 2441.8 1696.9 425 5865 2313.4 1369.6 332 6330 # tokens per section 149.6 64.8 12 447 147.8 56.7 33 298 145.8 58.6 24 290 # questions per story 36.8 28.9 5 161 44.5 29.5 13 100 43.7 28.8 12 107 # questions per section 2.8 2.440 0 18 2.9 2.3 0 16 3.0 2.4 0 15 # tokens per question 10.2 3.2 3 27 10.9 3.2 4 24 10.5 3.1 3 25 # tokens per answer 7.1 6.0 1 69 7.7 6.3 1 70 6.8 5.2 1 44 Table 3: Core statistics of the F airytale QA dataset, which has 278 books and 10580 QA-pairs. Categorization via Source of Answers Orthogonal to the aforementioned question categories, questions in F airytale QA are also categorized based on whether or not the answer source can be directly found in the text, namely explicit versus implicit questions. In general, explicit questions revolve around a specific story fact, and implicit questions require summarizing and making an inference based on information that is only implicit in the text. Using a combination of explicit and implicit questions yields an assessment with more balanced di ffi culty (Raphael, 1986; Zucker et al., 2010). In our data, explicit and implicit questions are defined as below (Examples in Appendix C):  Explicit questions ask for answers that can be directly found in the stories. In other words, the source of answer are spans of text. Implicit questions ask for answers that cannot be directly found in the text. Answering the questions require either reformulating language or making inference. In other words, the answer source is free-form, meaning that the answers can be any free-text, and there is no limit to where the answer comes from. Five annotators were involved in the annotation of QA pairs. All of these annotators have a B.A. degree in education, psychology, or cognitive science and have substantial experience in teaching and reading assessment. These annotators were supervised by three experts in literacy education. Annotation Guidelines The annotators were instructed to imagine that they were creating questions to test elementary or middle school students in the process of reading a complete story. We required the annotators to generate only natural, open-ended questions , avoiding yesor noquestions. We also instructed them to provide a diverse set of questions about 7 di ff erent narrative elements, and with both implicit and explicit questions. Each question in the dataset has a label on the narrative element / relation to be assessed and whether it is implicit or explicit. We asked the annotators to also generate answers for each of their questions. We asked them to provide the shortest possible answers but did not restrict them to complete sentences or short phrases. For explicit questions, annotators extracted the shortest phrase from the text as the answer (i.e., span). For implicit questions, annotators provided at least two possible answers for each question (i.e., free-form). We also asked the annotators to label which section(s) the question and answer was from. We did not specify the number of questions per story to account for story length variability and to allow annotators to create meaningful questions rather than be forced to add unnecessary questions. However, we did ensure that the annotators broadly averaged 2-3 questions per section in order to guarantee dataset size. Annotator Training and Cross-Checking All annotators received a two-week training in which each of them was familiarized with the coding template (described in the section below) and conducted practice coding on the same five stories. The practice QA pairs were then reviewed by the other annotators and the three experts, and discrepancies among annotators were discussed. At the end of the training session, the five annotators had a little disagreement with the questions generated by other coders. During the annotation process, the team met once every week to review and discuss each member's work. All QA pairs were cross-checked by two annotators, and 10% of the QA pairs were additionally checked by the expert supervisor. This process was to ensure that the questions focused on key information to the narrative and the answers to 451 Mean Min Max SD Story Characteristics Sections / story 14.7 2 60 9.2 Tokens / story 2196.7 228 7577 1401.3 Tokens / section 149.1 12 447 63.6 Question Characteristics Tokens / question 10.3 3 27 3.3 Tokens / answer 7.2 1 69 6.1 Questions / story 38.1 5 161 29 Questions / section 2.9 0 18 2.4 Table 4: Various descriptive statistics for the length of stories and number of questions in the dataset. Agreement among Annotators The questions generated by the five coders showed a consistent pattern. All coders' questions have similar lengths (average length ranging from 8 to 10 words among the coders) and have similar readability levels (aver-age readability between fourth to fifth grade among the coders). The distributions in narrative elements focused as well as implicit / explicit questions were also consistent. A detailed description of the distributions by coders is displayed in Appendix E. We chose not to use traditional inter-annotator agreement (IAA) metrics like Kappa coe ffi cients because we explicitly asked the coders to generate questions and answers with variable language to aid QA and QG models based on this dataset. This language variability leads to inaccurate IAA metrics by traditional means (Amidei et al., 2018), leading to our decision. Second Answer Annotation For the 46 stories used as the evaluation set, we annotate a second reference answer by asking an annotator to independently read the story and answer the questions generated by others. All questions were judged as answerable and thus answered by the second annotator. The second answers are used for both human QA performance estimation and for providing multiple references in automatic QA evaluation. We random split the F airytale QA dataset into train / val / test splits with a QA ratio of roughly 8:1:1. Table 3 shows the detailed statistics of the F airy tale QA Dataset in train / val / test splits. Overall, the resulting F airytale QA dataset contained 10,580 questions from 278 fairytale stories. The description of story and question characteristics is presented in Table 4. In F airytale QA, action and causal relationship questions are the two most common types, constituting 31.6% and 27.8%, respectively, of all questions. Outcome resolution, character, and feeling types each constitute about 10% of all questions. Setting and prediction questions are about 5% each. Our dataset contains about 75% explicit questions and 25% implicit questions (Table 5 for details). Validation of F airytale QA for Comprehension Assessment We validated the questions in F airy tale QA using established procedures in educational assessment development (zdemir and Akyol, 2019) and have proven that our questions have high reliability and validity. Specifically, we sampled a small subset of the questions in our dataset (11 questions generated for one story) and tested them among 120 students in prekindergartens and kindergartens. This study was preapproved by the IRB in first author's institution. The Cronbach's coe ffi cient alpha was 0.83 for the items in this story comprehension assessment; suggesting was high internal reliability. We also linked children's performance answering our questions to another validated language assessment (Mar-tin and Brownell, 2011), and the correlation was strong 0.76 (p < .001), suggesting an excellent external validity. In the following sections, we present a couple of baseline benchmarks on both the Question Answering (QA) task and the Question Generation (QG) task with F airytale QA. We leveraged both pretrained neural models and models fine-tuned on different QA datasets, including NarrativeQA and our dataset, F airytale QA. The baseline results show 452 Model Validation / Test ROUGE-L F1 Pre-trained Models BERT 0.104 / 0.097 DistilBERT 0.097 / 0.082 BART 0.108 / 0.088 Fine-tuned Models BART fine-tuned on NarrativeQA 0.475 / 0.492 BART fine-tuned on F airytale QA 0.533 / 0.536 Human  0.651 / 0.644 Table 6: Question Answering benchmarks on F airytale QA validation and test splits. that our F airytale QA demonstrates challenging problems to existing approaches, and those models fine-tuned on F airytale QA can benefit from the annotations a lot to achieve significant performance improvement. We also report human performance by scoring one reference answer to the other. Question Answering (QA) is a straightforward task that our F airytale QA dataset can contribute to. We leveraged the commonly-used Rouge-L F1 score for the evaluation of QA performances. For each QA instance, we compared the generated answer with each of the two ground-truth answers and took the higher Rouge-L F1 score. Here in Table 6, we show the QA performance of a few pretrained SOTA neural-model architectures: BERT (Devlin et al., 2018), BART (Lewis et al., 2019), and DistilBERT(Sanh et al., 2019). The quality of answers generated by these pre-trained models is on par with each other. Since BART outperformed other model architectures in the QA task of NarrativeQA (Mou et al., 2021), we decided to use BART as the backbone for our fine-tuned models. We report the performance of fine-tuned BART models with the following settings: BART fine-tuned on NarrativeQA, which is the SOTA model reported in (Mou et al., 2021), and another BART model fine-tuned on F airytale QA. We note that for the QA task, the model that was fine-tuned on F airytale QA dataset performs much better than the model fine-tuned on NarrativeQA by at least 5%. Even the human performance is underestimated Figure 1: Decomposed QA results (Rouge-L) on 7 narrative elements on the validation split. here because it is obtained via cross-estimation between two annotated answers, this result still leaves around 12% on both splits between human performance and the model fine-tuned with F airytale QA, which demonstrates that the QA task is still a challenging problem for existing works on our F airy tale QA dataset. We leave a full large-scale human study for evaluating the accurate human performance to future work. Performance Decomposition Given that F airy tale QA has question type annotations on all the question-answer pairs, it supports the decomposition of performance on di ff erent types, thus resulting in a comprehensive picture of which reading skills the models lack the most. Figure 1 presents the QA performance decomposition as a radar visualization. (The full results on both validation and test sets can be found in Table 10 in Appendix A). Compared to the model trained on NarrativeQA, our F airytale QA led to the biggest improvement on dimensions of Setting and Feeling with more than 10% in-crease. The Character and Prediction dimensions were also improved by a large margin (7-8%). The large improvements in these dimensions suggested that despite the NarrativeQA dataset's overall focus on narrative comprehension, it might not include questions that su ffi ciently cover some of the fundamental elements, probably due to the lack of detailed annotating protocol and typical crowd workers' limited knowledge in reading assessment. By comparison, on dimensions of Action , Causal Relationship and Outcome Resolution , our model fine-tuned on F airy tale QA resulted in smaller improvement compared 453 Figure 2: Learning curve of the QA model on F airytale QA with varying size of training data. to the model fine-tuned on NarrativeQA. This is likely due to the fact that most of the NarrativeQA questions are about event arguments and causal or temporal relations between events, as suggested by a human study (Mou et al., 2021). Our performance decomposition also revealed substantial gaps between existing SOTA models and humans. Specifically, humans were 15-20% better on Causal Relationship , Outcome Resolution and Prediction . The model-human performance gaps on Causal Relationship and Outcome Resolution likely reflected the deficiency of current NLP models in understanding story plots, and the gap on Prediction might be due to the fact that this dimension asked the models to envision what would come next in the text, which required connecting commonsense knowledge with the content of the text. The model-human performance gaps on Character and Setting were also considerable, suggesting that the models' ability to understand these basic reading elements still has much room for improvement. Finally, it was interesting that the model trained on our dataset outperformed humans on the Feeling dimension. This was likely because the answers to these Feeling questions were most explicitly described in the story. Therefore, it did not actually require reasoning of the character's mental states, but rather understanding which parts of the texts express the feelings. Another QA performance decomposition result based on explicit / implicit question types is provided in Appendix B. Learning Curve Finally, we present the learning curve of the BART QA model on our F airy tale QA. Figure 2 plots the model performance on the validation set with di ff erent sizes of training data. The curve became flatter after training with 6,000 QA pairs in our dataset. This suggested that Model Validation / Test ROUGE-L F1 BART fine-tuned on NarrativeQA 0.424 / 0.442 BART fine-tuned on F airytale QA 0.527 / 0.527 BART fine-tuned on NarrativeQA and F airytale QA 0.508 / 0.519 Table 7: Question Generation benchmarks on F airytale QA-validation and test splits. our dataset has a reasonably good size for fine-tuning a SOTA pre-trained model, and the performance gap between models and humans requires a more sophisticated reading model design rather than solely augmenting the training examples. In terms of the QG performance on F airytale QA, the task was to generate questions that correspond to the given answers and the context. This task has important empirical applications that in the future, models may help teachers to create questions in the educational settings. Similar to the QA task, we fine-tuned a BART model to generate a question conditioned on each human-labeled answer and corresponding story section. The generated question is then evaluated with the corresponding ground-truth question. We used ROUGE-L F1 score as the evaluation metric. For this QG task, we compare the models fine-tuned on NarrativeQA, on F airytale QA, and on both datasets. Table 7 displays the QG results. The model fine-tuned on F airytale QA demonstrated a clear advantage on Rouge-L over the model fine-tuned on NarrativeQA. It is worth noting that the model fine-tuned on both NarrativeQA and F airytale QA performs worse than the model fine-tuned on F airy tale QA only; we would assume that NarrativeQA 454 Input story section: the wild people who dwell in the south-west are masters of many black arts. they often lure men of the middle kingdom to their country by promising them their daughters in marriage, but their promises are not to be trusted. once there was the son of a poor family, who agreed to labor for three years for one of the wild men in order to become his son-in-law. Further analysis (Table 8) examined the distribution of generated question types according to the beginning word of a question (whwords). The questions generated by F airytale QA more closely resembled the pattern of the ground-truth questions, suggesting that our dataset was able to improve the model's ability to mimic the education experts' strategy of asking questions that assess the seven elements of reading comprehension. This result is further supported by qualitative analysis (as seen in examples in Table 9). Compared to the QG model trained with F airy tale QA, the baseline model trained with NarrativeQA dataset tended to generate vague questions that did not build upon specific contextual evidence within the narratives. These kinds of vague questions may not be suitable in educational settings, as improving students' skills to find text evidence to support their comprehension is a crucial aspect of reading education. The disparity between the two models might be attributed to how the QA-pairs were constructed in these two datasets: while NarrativeQA was constructed by crowd workers who only read the abstract of the stories, F airytale QA required annotators to read the complete story before developing QA-pairs. As such, it is not surprising that models trained on F airytale QA dataset could generate questions that are more closely related to the contextual evidence within the original text. In addition, we also observed that the model trained on NarrativeQA tended to generate questions with seemingly more correct grammar but were factually inaccurate (Table 12 Appendix C). In summary, we constructed a large-scale dataset, F airytale QA, for the context of children's narrative comprehension. The dataset was generated through a rigorous labeling process with educational domain experts. This dataset has been helpful to support preliminary works on QG tasks (Yao et al., 2022; Zhao et al., 2022) and already enabled possibilities for new downstream AI-for-Education applications (Zhang et al., 2022; Xu et al., 2021). Howerver, we acknowledge our work also has limitations that require future works to continue the exploration. As aforementioned, the human performance results for QA task are underestimated because they are obtained via cross-estimation between the two annotated answers. One possibility for future work is to conduct a large-scale human annotation to collect more answers per each question and then leverage the massive annotated answers to better establish a human performance evaluation. Another avenue of future work is to leverage our dataset to detect and remediate social stereotypes and biases represented in story narratives  the bias analysis in the children storybook corpus has been an underexplored research topic for the ML community, but it has profound societal impacts on the soceity. Through such analysis on our dataset, we may be able to answer how do social stereotype and bias come into a child's mind? In sum, there are many new research and application opportunities enabled by our F airytale QA dataset, and we welcome researchers from both NLP and education communities to join our e ff ort to continue this endeavor. We thank Schmidt Futures for providing funding for the development of the FairytaleQA dataset. This work is also supported by the National Science Foundation (Grant No. 1906321 and 2115382).
abstain abstain method abstain objective abstain result abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain objective method abstain abstain method objective result other result result other objective other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other abstain abstain abstain abstain method abstain abstain method abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain method method result abstain result abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method method abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result method abstain abstain abstain abstain abstain abstain abstain objective other other