To perform well on a machine reading comprehension (MRC) task, machine readers usually require commonsense knowledge that is not explicitly mentioned in the given documents. This paper aims to extract a new kind of structured knowledge from scripts and use it to improve MRC. We focus on scripts as they contain rich verbal and nonverbal messages, and two relevant messages originally conveyed by different modalities during a short time period may serve as arguments of a piece of commonsense knowledge as they function together in daily communications. To save human efforts to name relations, we propose to represent relations implicitly by situating such an argument pair in a context and call it contextualized knowledge. To use the extracted knowledge to improve MRC, we compare several fine-tuning strategies to use the weakly-labeled MRC data constructed based on contextualized knowledge and further design a teacher-student paradigm with multiple teachers to facilitate the transfer of knowledge in weakly-labeled MRC data. Experimental results show that our paradigm outperforms other methods that use weakly-labeled data and improves a state-of-the-art baseline by 4 . 3% in accuracy on a Chinese multiple-choice MRC dataset C 3 , wherein most of the questions require unstated prior knowledge. We also seek to transfer the knowledge to other tasks by simply adapting the resulting student reader, yielding a 2 . 9% improvement in F 1 on a relation extraction dataset DialogRE, demonstrating the potential usefulness of the knowledge for non-MRC tasks that require document comprehension. Given a question and a document, machine reading comprehension (MRC) tasks usually aim to select the correct answer option(s) from all options (Richardson et al., 2013; Lai et al., 2017) or extract a span as the answer from the document (Hermann et al., 2015; Rajpurkar et al., 2016). For many MRC tasks, which are mostly in multiple-choice formats, machine readers require extensive general world knowledge unstated in the given documents to perform well on tasks. Recent studies have showed the usefulness of other human-annotated MRC datasets (Chung et al., 2018) or data from relevant tasks such as natural language inference (Yin et al., 2021). There is also a trend towards taking advantage of existing crowdsourced general world knowledge graphs such as ConceptNet (Speer et al., 2017) or automatically constructed graphs (Zhang et al., 2020) to improve MRC tasks (Mostafazadeh et al., 2016; Ostermann et al., 2018; Sun et al., 2019a; Huang et al., 2019). However, most progress has been limited to English, and it is expensive, time-consuming, laborious, and error-prone to construct clean, large-scale datasets of MRC-related tasks or general world knowledge graphs, which are not always available even for high-resource languages such as Chinese. This paper aims to extract a new kind of structured general world knowledge from external unstructured Chinese corpora, which is seldom studied, and investigate the use of the knowledge-based augmented data to improve MRC. Typically, each piece of general world knowledge is represented as a triple that contains two phrases (e.g., ( finding a lost item , happiness ) and the relation (e.g., CAUSES ) between phrases, which can be one of a small pre-defined set of relations (Tandon et al., 2014). However, naming relations often requires substantial human efforts (e.g., guidelines, annotations, or relation-specific patterns), and it is also unclear whether we need to explicitly represent relations if the final goal is to improve downstream tasks that do not directly depend on the reliability of relations in triples from 8736 other sources. Once we have decided not to name relations, one natural question is whether we could implicitly represent relations between two phrases. We suggest that adding context in which the phrases occur may be useful as such a context constrains the possible relations between phrases without intervening in the relations explicitly (Brzillon et al., 1998). Hereafter, we call a triple that contains a phrase pair and its associated context as a piece of contextualized knowledge for convenience. Besides verbal information that is written or spoken, it is well accepted that nonverbal information is also essential for face-to-face communication (Jones and LeBaron, 2002). We regard related verbal and nonverbal information as the phrase pair; we treat the context in which the verbal-nonverbal pair occurs as the context. Such triples contain rich commonsense knowledge as verbal and nonverbal information function together in communications, and this kind of knowledge is assumed to be known by most people without being formally taught, just as commonsense knowledge. For example, as shown in Table 1, the pause in I'm going......to his house. is related to thinking , the internal state of the speaker. We suggest film and television show scripts are good source corpora for extracting contextualized knowledge as they contain rich strongly interrelated verbal (e.g., utterances of speakers) and nonverbal information (e.g., body movements, vocal tones, or facial expressions of speakers), which is originally conveyed in different modalities within a short time period and can be easily extracted from the scripts. Furthermore, a script usually contains multiple scenes, and the entire text of the scene from which the verbal-nonverbal pair is extracted can serve as the context. According to the relative position of a verbal-nonverbal pair in a scene, we create four types of simple lexical patterns to extract contextualized verbal-nonverbal knowledge (Section 2). To use contextualized knowledge to improve MRC, we randomly select nonverbal messages from the same script as distractors to convert each piece of knowledge into a weakly-labeled MRC instance (Section 3). We explore different two-stage fine-tuning strategies to use the weakly-labeled MRC data: for example, we first train a model on the combination of the weakly-labeled data and the target MRC data that is human-annotated but small-scale, and then, we fine-tune the resulting model on the target data (Section 4). However, we observe that increasing the amounts of weakly-labeled data does not lead to noticeable gains than using one subset of the data. Inspired by teacher-student studies that facilitate the transfer of clean knowledge (Li et al., 2014; Hinton et al., 2015; You et al., 2019), we further design a teacher-student paradigm with multiple teachers, each trained with a subset of the hard-labeled, weakly-labeled data, and we feed soft-labeled, weakly-labeled/clean data to a student in a two-stage fashion, similar to the two-stage fine-tuning strategy mentioned earlier (Section 5). We evaluate our method on C 3 , a representative multiple-choice general-domain MRC dataset for Chinese wherein most questions require commonsense knowledge beyond the given contents (Sun et al., 2020). To date, C 3 is the only public Chinese dataset of this task. Experimental results show that our multi-teacher paradigm leads to +4 . 3% in accuracy over a state-of-the-art baseline (Cui et al., 2020) and also outperforms other methods such as pre-training upon all scripts and hard-label fine-tuning that uses the same structured knowledge. Also, under the same framework, constructing weakly-labeled MRC data based on verbal-nonverbal knowledge leads to bigger gains compared to other strong data augmentation strategies (and some are even based on human-annotated commonsense knowledge graphs (Speer et al., 2017)). Finally, we seek to transfer the knowledge to help other tasks by adapting the final student MRC model, yielding +2 . 9% in F 1 on the Chinese set of a document-level relation extraction dataset DialogRE (Yu et al., 2020) over competitive baselines. The main contributions are as follows: ( i ) we suggest that scripts can be a good resource for extracting rich contextualized verbal-nonverbal knowledge with minimal supervision; ( ii ) we explore and compare several fine-tuning strategies to use weakly-labeled data based on the extracted knowledge and further propose a teacher-student paradigm with multiple teachers to better learn from large-scale weakly-labeled data; and ( iii ) our empirical results demonstrate the effectiveness of the multi-teacher paradigm and the usefulness of contextualized verbal-nonverbal knowledge for MRC tasks that require commonsense knowledge. Understanding the interactions between verbal and nonverbal information in communications requires prior knowledge, and this knowledge is assumed 8737 to be known by most people without being taught, just as commonsense knowledge. We propose to use interrelated verbal and nonverbal information as phrases in the classical triple-style knowledge representation and situate them in a context. Formally, we call a triple ( v , c , n ) as a piece of contextualized knowledge , containing a pair of related verbal information v and nonverbal information n , and the associated context c . We choose to extract contextualized knowledge from film and television show scripts as plentiful verbal and nonverbal messages frequently co-occur in scripts, and they can be easily separated. Scenes in a script are delimited by blank lines. Based on the relative position of verbal and nonverbal information (intra-turn or cross-turn), we extract four subsets of contextualized knowledge (B c , B n , I, and O):  Beginning: The nonverbal information n appears after a speaker name and before the speaker's utterance. We regard the speaker name and the corresponding utterance as v . Clean (B c ): We only extract nonverbal information n within parentheses. Noisy (B n ): The first span of a turn, followed by a colon, can also contain both a speaker name and nonverbal information about this speaker. For example, from Xiaocong Le took the cup of hot water: Thank you!'  , Xiaocong Le will be recognized as the speaker name; took the cup of hot water will be treated as n . We roughly regard a phrase as a speaker name if it appears in the first span of other turns in the same scene. Inside (I): We only extract nonverbal information n enclosed in parentheses, which appears within an utterance. All the information in the same turn except n is treated as v . Outside (O): Here n is an action line that mainly describes what can be seen or heard by the audience, marked by (cid:51) in Table 1. We regard the turn (if it exists) before the action line as its corresponding v . We do not extract phrases in parentheses or action lines as nonverbal information if they are terminologies for script writing such as O.S." , V.O." CONT'D , beat , jump cut" , and fade in . 1 All contextualized knowledge triples extracted from a scene share the same context c , i.e., the scene itself. We do not exploit scene headings (e.g., when and where a scene takes place) (marked by (cid:50) in Table 1), as they are intentionally designed to cover the content of a whole scene, which is already used as context. We leave other types of patterns (e.g., using the turn after the action line to construct O) for future studies. Due to space limitations, we list more extracted contextualized verbal-nonverbal knowledge triples in Appendix A.3 (Table 11). 1 We will release the stop word list along with the code at https://github.com/nlpdata/script . We mainly discuss how to convert the extracted triples into multiple-choice instances and leave its extension to other types (e.g., extractive or abstractive) of MRC tasks for future research. We generate one instance for each piece of contextualized knowledge. For each triple ( v , c , n ), we remove n from context c , and we regard the remaining content as the reference document, verbal information v as the question, and the nonverbal information n as the correct answer option. To generate distractors (i.e., wrong answer options), we randomly select N items from all the unique nonverbal messages in other triples, which are extracted using the same type of patterns from the same script as ( v , c , n ). Note that although we only generate one instance based on each triple, it is easy to generate more instances by changing distractors. We aim to use the constructed weakly-labeled data to improve a downstream MRC task. Given weakly-labeled data generated based on contextualized knowledge extracted from scripts, we first use the weakly-labeled data in conjunction with the training set of the target MRC data as the training data to train the model and then fine-tune the resulting model on the target MRC data as illustrated in Figure 1. We do not adjust the ratio of clean data to weakly-labeled data observed during training as previous joint training work on other tasks such as machine translation (Edunov et al., 2018). Another way is to perform separate training (or sequential transfer learning (Ruder et al., 2019)): we can first train the model on the weakly-labeled data and then fine-tune it on the target data. We assume that using the small-scale, clean data to guide weakly-supervised training is more helpful, and we compare joint and separate training in Section 6.5. In our preliminary experiment, we observe that increasing the amounts of weakly-labeled data does not lead to noticeable gains than using one subset of the weakly-labeled data. Inspired by previous teacher-student frameworks (You et al., 2019; Wang et al., 2020) that train a multi-domain student model with multiple teacher models to help knowledge transfer, and each teacher model is trained on a clean domain-specific subset, we extend the supervised fine-tuning weakly-supervised fine-tuning MRCmodel weakly-labeleddata labeleddata knowledgeextraction MRCmodel labeleddata distractorgeneration unstructuredcorpus Figure 1: Two-stage fine-tuning framework overview (hard labels are used during both stages). multi-teacher idea to let models learn better from large-scale weakly-labeled MRC data for improving a target MRC task. As introduced in Section 3, we can have multiple subsets of weakly-labeled data generated based on verbal-nonverbal knowledge extracted by different types of patterns ((e.g., B c or I) in (Section 2)). Let V denote a set of labeled instances, W 1 , . . . , W (cid:96) denote (cid:96) sets of weakly-labeled instances, and W = (cid:83) 1  i  (cid:96) W i . For each instance t , we let m t denote its total number of answer options, and h ( t ) be a hard label vector (one-hot) such that h ( t ) j = 1 if the j -th option is labeled as correct. We train (cid:96) teacher models, denoted by T 1 , . . . , T (cid:96) , and optimize T i by minimizing (cid:80) t  V  W i L 1 ( t,  T i ) . L 1 is defined as L 1 ( t,  ) =  (cid:88) 1  k  m t h ( t ) k log p  ( k | t ) , where p  ( k | t ) denotes the probability that the k -th option of instance t is correct, estimated by the model with parameters  . We define soft label vector s ( t ) such that s ( t ) k =    h ( t ) k + (1   ) (cid:88) 1  j  (cid:96) 1 (cid:96) p  T j ( k | t ) t  V  h ( t ) k + (1   ) p  T i ( k | t ) t  W i , where   [0 , 1] is a weighting parameter, and k = 1 , . . . , m t . We then train a student model, denoted by S , still in a two-stage fashion. In stage one (i.e., weakly-supervised fine-tuning), we optimize S by minimizing (cid:80) t  V  WL 2 ( t,  S ) , where L 2 is defined as L 2 ( t,  ) =  (cid:88) 1  k  m t s ( t ) k log p  ( k | t ) . minimizing (cid:80) t  VL 2 ( t,  S ) . See Figure 2 for an overview of the multi-teacher paradigm. We collect 100,46 scripts in Chinese, and most of them are intended for films and television shows. 2 After segmentation and filtering, we obtain 199,280 scenes, each of which contains at least one piece of contextualized knowledge. We generate four subsets of weakly-labeled MRC data based on contextualized knowledge extracted by four types of patterns. For comparisons, we use existing human-annotated triples about commonsense knowledge in the Chinese version of ConceptNet (Speer et al., 2017) as well as span-style augmented multiple-choice MRC data (Sun et al., 2019b) extracted from all scripts. We set the number of distractors N to five for weakly-labeled MRC instances. For evaluation, we use C 3 , so far as we know, the only public multiple-choice MRC data for Chinese with a focus on commonsense knowledge. About 86 . 8% of questions in C 3 involve prior knowledge (i.e., linguistic, domain-specific, and commonsense knowledge) unstated in the given texts, and all instances are carefully designed by experts such as 2 https://www.1bianju.com . second-language teachers. Each instance consists of a document, a question, and multiple answer options; only one option is correct. In addition, we use the human-annotated Chinese set of a bilingual relation extraction dataset DialogRE, which also requires document-level understanding to predict relations from 36 possible types between an argument pair. 3 We provide data statistics in Table 2. While we focus on resources (e.g., scripts and datasets) in Chinese, our extraction and training methods are not limited to a particular language. We follow Sun et al. (2020) for the model architecture consisting of a pre-trained language model and a classification layer on top of the model. We use RoBERTa-wwm-ext-large (Cui et al., 2020) as the pre-trained language model, which achieves state-of-the-art performance on C 3 and many other natural language understanding tasks in Chinese (Xu et al., 2020). We leave the exploration of very recent pre-trained language models such as ERNIE 3.0 (Sun et al., 2021) for future work. When the input sequence length exceeds the limit, we repeatedly discard the last turn in the context, or the first turn if the last turn includes the extracted verbal information. For all our implementations and methods, we train a model for one epoch during the weakly-supervised fine-tuning stage and eight epochs during the supervised fine-tuning stage; other parameters remain unchanged. We set  (de-fined in Section 5) to 0 . 5 in all experiments based on the rationale that we can make best use of the soft labels while at the same time making sure arg max k s ( t ) k is always the index of the correct option for instance t . Carefully tuning  on the development set may lead to further improvements, 3 https://github.com/nlpdata/dialogre . Table 3 reports the main results. The baseline accuracy ( 73 . 4% {T0}) is slightly lower than previously reported using the same language model 4 as we report the average accuracy over five runs with different random seeds for all our supervised fine-tuning results. For easy reference, we indicate the index for each result in curly brackets in the following discussion. Two-Stage Fine-Tuning : The performance of a model after the first fine-tuning stage over the combination of the C 3 dataset and much larger weakly-labeled data is worse (e.g., 71 . 6% {T1}) than baseline performance ({T0}). Further fine-tuning the resulting model on the C 3 dataset consistently leads to improvements (e.g., 74 . 0% {3}) over the baseline {T0}, demonstrating the necessity of supervised fine-tuning when large-scale weakly-labeled data is used. In particular, each subset of the weakly-labeled data based on contextualized knowledge extracted by a certain type of patterns can boost the performance ({3, 4, 5, 6}); the magnitude of accuracy improvement is 1 . 2% on average. Multi-Teacher Paradigm : When we combine all the weakly-labeled data during weakly-supervised fine-tuning, the performance gain after the supervised fine-tuning stage ( 75 . 2% {2}) is not as impressive as expected, given the best performance 4 https://github.com/CLUEbenchmark/CLUE . achieved by only using one subset ( 74 . 9% {5}). As a comparison, our teacher-student paradigm with multiple teacher models (i.e., {T1, T2, T3, T4}) trained with different subsets of weakly-labeled data leads to up to 3 . 7% improvement in accuracy ({10} vs. {3, 4, 5, 6}). The advantage is reduced but still exists even when we use the original hard labels instead of soft labels in the second fine-tuning stage ( 76 . 4% {M1}). To examine the contribution of the contextualized knowledge, under the exact same multi-teacher paradigm, we train four {T0}-teachers only on the target MRC dataset with different random seeds, and let them teach the student. Such multi-teacher baselines ({M2M4}) perform much worse than our proposed method {10}, even under-performing the original {T0}. Pre-Training or Fine-Tuning : We also consider continuing pre-training the pre-trained language model (i.e., RoBERTa-wwm-ext-large) over the plain texts of the scripts using the masked language model (MLM) objective (Devlin et al., 2019; Liu et al., 2019), which might benefit more downstream tasks without requiring task-specific data augmentation. We follow previous work (Cui et al., 2020; Xu et al., 2020) to conduct whole word masking to generate training instances and continue pretraining the model for 200 K steps considering the much smaller size of the scripts ( 0 . 1 B tokens) compared with that of the original pre-training data ( 5 . 4 B tokens). 5 As shown in {1} in Table 3, con-5 Here, increasing the number of steps further hurts MRC. tinued pre-training over the scripts has a slight negative impact on MRC performance. This result is perhaps not too surprising as many studies demonstrate the usefulness of continued pre-training only when the size of domain-specific unlabeled data (e.g., NEWS (Zellers et al., 2020) that contains 6 . 7 B tokens) is sufficient to update model parameters using MLM (Gururangan et al., 2020). Considering the size of scripts, to improve the performance on a general-domain MRC task, we argue that it is more effective to inject the knowledge during the fine-tuning stage, using the same training objective as that of the target MRC task. From the perspective of improving a target MRC task, we first compare the extracted contextualized verbal-nonverbal knowledge with crowdsourced commonsense knowledge graphs. As most of such graphs are in English, we only compare the knowledge with the crowdsourced Chinese version of ConceptNet. To convert each triple (subject, relation type, object) in ConceptNet into an MRC instance, we use the subject as question and object as answer, and we experiment with two types of documents: (i) leave the document empty and (ii) use the relation type as the document. We randomly select phrases in ConceptNet other than the phrases in each triple as distractors. For a fair comparison, we compare (ii) with baseline {2} in Table 3 using the same two-stage hard-label fine-tuning strategy. The numbers of weakly-labeled instances based on contextualized knowledge and ConceptNet are close (Table 2). The results reveal that under the two-stage fine-tuning framework, introducing ConceptNet yields up to +0 . 7% in accuracy, but using contextualized knowledge gives a bigger gain of 1 . 8% in accuracy. It is worth mentioning that keeping context (i.e., relation types) for ConceptNet-based weakly-labeled instances is helpful for downstream tasks ({9} vs. {8}). Based on all the scripts, we also implement a span-style multiple-choice MRC data augmentation method (Sun et al., 2019b), which randomly concatenates several sentences in a scene as a question, randomly removes spans from the question, and concatenates them to form an answer. Distractors are generated by randomly replacing spans in the answer with randomly selected spans from the same scene. This method ({7}) does not lead accuracy gains over the baseline {T0}, supporting the usefulness of augmenting MRC data based on verbal-nonverbal knowledge from scripts. We have shown that the present multi-teacher paradigm helps learn better from large-scale weakly-labeled data. We further conduct ablation studies to examine critical factors for this paradigm. All other aspects remain the same as {10} in Table 3. We remove the context (i.e., scene) from each instance in the weakly-labeled data and leave it empty. We also experiment with removing C 3 from the weakly-supervised fine-tuning stage when we train teacher and student models and only use hard-labeled C 3 during the final supervised fine-tuning stage. As shown in Table 4, accuracy decreases in both cases. For two-stage hard-label fine-tuning {2}, keeping context is still helpful, while including C 3 during weakly-supervised fine-tuning seems less important. These results show the usefulness of keeping the context of weakly-labeled data and involving the small-scale, human-annotated target MRC data to guide weakly-supervised fine-tuning for the multi-teacher paradigm. It is difficult, however, to infer which pieces of knowledge help the improved MRC instances. As an alternative solution, we study the impacts of the contextualized knowledge on different types of questions based on the annotated subset ( 300 instances) released along with the dataset. As shown in Table 6, our method generally improves performance on all types of questions, especially those that require commonsense knowledge. In particular, we see +10 . 0% in accuracy or more on questions that require CAUSE-EFFECT , SCENARIO , or PART-WHOLE , three subcategories of commonsense knowledge. For instance, given a conversation, Female: Sir, can you drive faster? I'm afraid that I will be late for the exam. Male: No, the speed is already quite fast. Safety is also very 8742 Scene 2 Lili Kang: Hurry up, the Central Hospital! The child just fell. Hurry up! (cid:51) The car is going fast. Lili Kang: Hurry up! hurry up! Taxi driver: I can't go any faster, I'll be fined by the police if I go any faster. (cid:51) The taxi driver said and turned to look at the child wrapped in the blanket. Taxi driver ( surprised ): This kid's eyes are wide open, and he is in high spirits. Where did he break it? [...] pattern type nonverbal verbal B c surprised Taxi driver: This kid's eyes are wide open, and he is in high spirits. Where did he break it? Table 5: An taxi-related example of extracted contexutalized knowledge ([...]: words omitted; (cid:51) : action line). important." , we require scenario knowledge about activities of humans, their corresponding location information, and personal information such as the profession, in order to answer the question about the possible location ( taxi ) of the two speakers. It is possible that the temporal order between verbal and nonverbal information sometimes indicates causality. Also, the scenes used as context in the structured knowledge contain rich scenario knowledge. For example, similar to the scenario in the previous MRC instance, a taxi driver also refuses the passenger's request to drive faster in one of our extracted triples (Table 5). See more improved MRC instances and analysis in Appendix A.1. Instead of converting the extracted triples into weakly-labeled relation extraction instances and training from scratch, we simply replace the classification layer of an MRC model with a multi-class multi-label classification layer following the baseline released by Yu et al. (2020) and fine-tune the whole architecture on DialogRE. We compare the performance of methods that use different weights for parameter initialization except for the randomly initialized classification layer. We achieve +2 . 9% in F 1 and +3 . 1% in F 1 c on DialogRE (Table 7). The metric F 1 c is used to encourage a model to identify relations between arguments as early as possible rather than after reading the whole dialogue. Introducing C 3 alone also allows us to achieve a slight gain over the relation extraction baseline. It might be interesting to investigate the relevance between document-level relation extraction and MRC for further performance boost. We also conduct preliminary experiments by fine-tuning the MRC student on multiple-choice translated MRC datasets DREAM (Sun et al., 2019b) and Cosmos QA (Huang et al., 2019) (orig-inally in English) and see at least +2 . 7% in accuracy. As this is not the main focus of the paper, we provide more details in Appendix A.2. A common solution to obtain external contextualized knowledge is to utilize existing knowledge bases via distant supervision (Ye et al., 2019). We extract contextualized knowledge from scripts, wherein contexts (i.e., scenes) are naturally aligned with verbal-nonverbal pairs to avoid noise. Besides, we focus on verbal-nonverbal knowledge to improve MRC, which is seldom studied. Our work is also related to commonsense knowledge extraction, which relies on human-annotated triples (Xu et al., 2018; Bosselut et al., 2019), high-precision syntactic/semantic patterns (Zhang et al., 2020; Zhou et al., 2020) specific to each relation, 8743 or lexical databases (Tandon et al., 2014, 2015). In contrast, we skip the step of naming relations between phrases and situate structured knowledge in its context. We extract knowledge based on language-independent, scenario-independent lexical patterns, which do not require dependencies such as training data, a broad-coverage semantic lexicon, or a high-quality syntactic parser. Our teacher-student paradigm is most related to multi-domain teacher-student training for automatic speech recognition (You et al., 2019) and machine translation (Wang et al., 2020). Instead of clean domain-specific data, each of our teachers is trained with weakly-labeled data . Due to the introduction of large-scale weakly-labeled data, we use the training data (hard/soft labels) of the target MRC task in all stages to guide the training. The model architecture of students is the same as that of teachers to improve the task performance, which is different from previous knowledge distillation studies (Li et al., 2014; Hinton et al., 2015). This paper introduces how to extract contextualized verbal-nonverbal knowledge from scripts and use the knowledge to improve multiple-choice MRC. We compare different strategies to use the knowledge-based weakly-labeled data and further propose a multi-teacher paradigm to better learn from large-scale weakly-labeled data. Experimental results show the effectiveness of multi-teacher and usefulness of the knowledge for different tasks. We would like to thank the anonymous meta-reviewers/reviewers for their insightful feedback. We would like to thank Prof. Ralph Grishman from New York University for his valuable suggestions.
abstain objective method objective result result abstain result abstain abstain abstain abstain abstain abstain abstain objective abstain objective abstain method abstain abstain abstain abstain abstain result abstain method result objective result method method abstain result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method abstain abstain method abstain method method method other method method method method method abstain other abstain method abstain other abstain method abstain abstain method abstain method method method abstain abstain result result abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain other abstain result result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result result abstain other abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain result abstain abstain other method abstain abstain objective method abstain method method other result objective abstain other other