Fine-grained entity typing (FGET) aims to classify named entity mentions into fine-grained entity types, which is meaningful for entity-related NLP tasks. For FGET, a key challenge is the low-resource problem  the complex entity type hierarchy makes it difficult to manually label data. Especially for those languages other than English, human-labeled data is extremely scarce. In this paper, we propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages. Specifically, we use multi-lingual pre-trained language models (PLMs) as the backbone to transfer the typing knowledge from high-resource languages (such as English) to low-resource languages (such as Chinese). Furthermore, we introduce entity-pair-oriented heuristic rules as well as machine translation to obtain cross-lingual distantly-supervised data, and apply cross-lingual contrastive learning on the distantly-supervised data to enhance the backbone PLMs. Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages, even without any language-specific human-labeled data. Our code is also available at https://github.com/thunlp/CrossET. Recently, various efforts have been devoted to exploring fine-grained entity typing (FGET) (Ling and Weld, 2012; Li et al., 2020), aiming to identify concrete fine-grained entity types for named entity mentions in sentences (Figure 1). Since the type information of named entity mentions is useful for understanding textual semantics, FGET is widely applied to enhance entity-related tasks, such  Corresponding authors. as coreference resolution (Khosla and Rose, 2020), entity linking (Onoe and Durrett, 2020; Chen et al., 2020a), relation extraction (Ren et al., 2017; Zhou and Chen, 2021) and event extraction (Nguyen et al., 2016; Yang and Mitchell, 2016). Despite the success of FGET, the low-resource problem is always a challenge of FGET, since the complex type hierarchy makes it difficult to manually label data. To alleviate the low-resource problem, besides utilizing auto-labeled data (Ling and Weld, 2012; Gillick et al., 2014; Xin et al., 2018; Dai et al., 2021), manually building FGET datasets is the most effective approach (Sang and De Meulder, 2003; Hovy et al., 2006; Ling and Weld, 2012; Choi et al., 2018; Ding et al., 2021). However, existing FGET datasets are mainly in English. For datasets in specific languages other than English, such as Chinese (Lee et al., 2020), Japanese (Suzuki et al., 2016), Dutch and Spanish (van Erp and Vossen, 2017), their scale and quality are not comparable to those English datasets. In this paper, we introduce a cross-lingual framework to learn FGET models for low-resource languages, via utilizing the data in high-resource languages 2241 (e.g. utilizing English datasets). Transferring the typing knowledge from high-resource languages to low-resource languages is not easy. As different languages have quite different patterns, it is challenging to understand the semantics of both high-resource and low-resource languages at the same time. With only a few examples of low-resource languages and no parallel data, it is also hard to bridge different languages for knowledge transfer. To handle these issues: (1) we use multi-lingual pre-trained language models (PLMs) as backbone. Multi-lingual PLMs such as M-BERT (Devlin et al., 2019) are pre-trained on large-scale multi-lingual corpora, taking it as the backbone can well encode data in different languages into the same semantic space (Han et al., 2021). (2) we apply heuristic rules and cross-lingual contrastive learning to bridge multiple languages. We design several entity-pair-oriented heuristic rules to obtain distant supervision, which can automatically annotate entity types by utilizing latent relations between entity pairs. Machine translation is used on the auto-labeled data to establish a connection between high-resource and low-resource languages. Finally, we apply contrastive learning to learn similarities between cross-lingual auto-labeled types, instead of using pseudo-labels to learn a classifier, which can enhance the type recognition ability and reduce the side effect of auto-labeled data. For convenience, we name our cross-lingual contrastive learning framework C ROSS -C in the following sections. We conduct experiments on two popular FGET datasets: Open-Entity (Choi et al., 2018) and Few-NERD (Ding et al., 2021), and translate their test sets into non-English versions to evaluate the effectiveness of CROSS-C for low-resource languages. Quantitative experimental results show that applying CROSS-C can easily train effective FGET models for low-resource languages, even without any language-specific human-labeled data. Besides quantitative experiments, we also provide some visualization of feature spaces and conduct case studies for qualitative analysis to show how CROSS-C works. In this section, we will introduce our cross-lingual framework to learn FGET models for low-resource languages. We will first give some essential notations and definitions, and then elaborate on the details of our framework. As shown in Figure 1, given a sentence x and one named entity mention m in the sentence, our goal is to determine types from a fine-grained type set T according to the sentence context for the mention m . Note that FGET is a multi-label classification problem, since multiple types can be assigned to a single named entity mention. For a high-resource language h , sufficient human-labeled data {X h , Y h } exists, where X h = { x h, 1 , x h, 2 , . . . } is the sentence set and Y h = { y h, 1 , y h, 2 , . . . } is the label set. Each sentence x h,i  X h contains a named entity mention m h,i , and y h,i  T is the fine-grained type set of the named entity mention m h,i . Similarly, we define the dataset {X l , Y l } for a low-resource language l , where |X l |  |X h | 1 . In this paper, we use {X h , Y h } , {X l , Y l } and large-scale unlabeled multi-lingual data to train a FGET model for the low-resource language l . We use multi-lingual BERT (M-BERT) (Devlin et al., 2019) as the framework backbone to encode the input. M-BERT has the same architecture as BERT, but is pre-trained on the multi-lingual corpora in 104 languages. Therefore, M-BERT has a good ability to transfer knowledge across languages (Pires et al., 2019; Selvaraj et al., 2021), making it suits our setting well. Note that, our framework does not depend on a specific PLM, any other multi-lingual PLMs can also be used as the backbone to encode the input. Given a sentence x = [ w 1 , . . . , m, . . . , w n ] , where m is the named entity mention, we additionally insert an entity marker [ENT] on each side of the mention m . By feeding the sentence with entity markers into M-BERT, we can get representations [ h w 1 , . . . , h [ENT] , h m , h [ENT] , . . . , h w n ] for all input tokens. The left entity marker representation h [ENT] is used to represent the named entity mention. For simplicity, we denote this process as m = M-PLM ( x ) , where m is the entity mention representation and x is the input sentence. Given each entity type t  T , the probability that the mention m in the sentence x can be classified as 1 In our experiments, we focus on handling a difficult and extreme case |X l | = 0 , i.e, there is no any human-labeled data for the low-resource language l . the type t is given as P  ( t | x ) =  (cid:0) t  M-PLM ( x ) (cid:1) , (1) where  is the sigmoid function, t is the representation of the entity type t , and  indicates all learnable model parameters. With the data {X h , Y h } in the high-resource language h and the data {X l , Y l } in the low-resource language l , the overall optimization objective is as arg max  (cid:104) L high (  ) + L low (  ) (cid:105) , (2) where L high (  ) and L low (  ) respectively indicate the loss functions for the high-resource language h and the low-resource language l . These loss functions are defined as L high (  ) = 1 |X h | |X h | (cid:88) i =1 (cid:88) t T (cid:2)  t  y h,i log P  ( t | x h,i ) + (1   t  y h,i ) log(1  P  ( t | x h,i )) (cid:3) , L low (  ) = 1 |X l | |X l | (cid:88) i =1 (cid:88) t T (cid:2)  t  y l,i log P  ( t | x l,i ) + (1   t  y l,i ) log(1  P  ( t | x l,i )) (cid:3) . (3) For the function  c , if the condition c is satisfied, then  c = 1 , otherwise  c = 0 . As we mentioned before, there are only a few human-labeled examples in low-resource languages. Although multi-lingual PLMs can provide an effective backbone to understand multilingual semantics, more examples are still required to bridge different languages. The existing distantly-supervised methods annotate the mentions of the same entity in multiple sentences with the same pseudo label (Ling and Weld, 2012; Gillick et al., 2014; Xin et al., 2018). However, in Figure 2, the mention Mark Twain requires to be annotated with writer or miner according to specific semantics. Hence, these single-entity-oriented heuristic rules inevitably bring much noise. To this end, we introduce heuristic rules orienting entity pairs to automatically annotate data with less noise. Instead of annotating specific entity types, we annotate whether two named entity mentions are of similar types. On the one hand, this strategy can consider the correlation and similarity between different types. On the other hand, this strategy is suitable for contrastive learning, which can reduce the side effect of data noise. In fact, in relation extraction, recent works have adopted similar strategies (Soares et al., 2019; Peng et al., 2020) and achieved promising results. More specifically, as shown in Figure 2, we take three rules to obtain distantly-supervised data: (1) Rules without knowledge bases . As shown in Figure 2(a), without using knowledge bases, if one entity pair is mentioned by two sentences, the mentions of the same entity in these two sentences are considered to have similar types. (2) Rules with knowledge bases . As shown in Figure 2(b), by using knowledge bases, if entity pairs in two sentences have same relations in knowledge bases, and these pairs have shared entities, the mentions of corresponding entities are considered to have similar types. (3) Building cross-lingual data with machine translation . As shown in Figure 2(c), we use machine translation to translate the data from the high-resource language to the low-resource language. Owing to the translation, the above-mentioned auto-labeled examples and their translated versions constitute a cross-lingual distantly-supervised dataset. By making full advantage of distant supervision and machine translation, we can greatly expand our dataset to bridge high-resource and low-resource languages, and further transfer the typing knowledge between these languages. To make FGET models pay more attention to textual contexts rather than merely focusing on entity names, we use the [MASK] token to mask named entity mentions with a probability of 0 . 5 . With all above-mentioned heuristic rules in Section 2.3, we can get the distantly-supervised data  X h = {  x h, 1 ,  x h, 2 , . . . } in the high-resource language h , the distantly-supervised data  X l = {  x l, 1 ,  x l, 2 , . . . } in the low-resource language l , and the translated data  X t = {  x t, 1 ,  x t, 2 , . . . } Given any two sentences x 1 , x 2 in these distantly-supervised datasets, we use the function s ( x 1 , x 2 ) to measure the similarity between the entity mentions of the two sentences. In practice, we take the cosine similarity with temperature  as the function s ( x 1 , x 2 ) : s ( x 1 , x 2 ) = M-PLM ( x 1 )  M-PLM ( x 2 )  M-PLM ( x 1 )    M-PLM ( x 2 )    , (4) where M-PLM (  ) is the entity mention representation computed by multi-lingual PLMs. The cross-lingual contrastive learning consists of two important objectives. One is the mono-lingual objective for each language, and the other is the cross-lingual objective. For both the high-resource language h and the low-resource language l , their mono-lingual objectives are defined as follows, L mono-h (  ) = 1 |  X h | |  X h | (cid:88) i =1 (cid:2) log (cid:88)  p P ( x h,i ) e s ( x h,i ,  p )  log( (cid:88)  p P ( x h,i ) e s ( x h,i ,  p ) + (cid:88)  n N ( x h,i ) e s ( x h,i ,  n ) ) (cid:3) , L mono-l (  ) = 1 |  X l | |  X l | (cid:88) i =1 (cid:2) log (cid:88)  p P ( x l,i ) e s ( x l,i ,  p )  log( (cid:88)  p P ( x l,i ) e s ( x l,i ,  p ) + (cid:88)  n N ( x l,i ) e s ( x l,i ,  n ) ) (cid:3) , (5) where P ( x h,i )   X h and N ( x h,i )   X h are respectively the positive set and the negative set of the example  x h,i . P ( x l,i ) and N ( x l,i ) are defined in a similar way for the example  x l,i . To ensure that the model does not push the representations of different languages far away, so that the low-resource language l can benefit from the high-resource language h , we further use  X h and its translated set  X t to define the cross-lingual objective as follows, L cross (  ) = 1 |  X t | |  X t | (cid:88) i =1 (cid:2) log (cid:88)  p P ( x t,i ) e s ( x t,i ,  p )  log( |  X t | (cid:88) j =1  i  = j e s ( x t,i ,  x t,j ) + |  X h | (cid:88) j =1 e s ( x t,i ,  x h,j ) ) (cid:3) , (6) where P ( x t,i )   X h   X t is the positive set of the example  x t,i . arg max  [ L cross (  ) + L mono-r (  ) + L mono-l (  )] . We divide the whole learning process into two stages: pre-training and fine-tuning. The pretraining stage is to use Eq. (7) to optimize parameters on the distantly-supervised data. Considering computational efficiency, every time we sample a batch of examples for contrastive learning, and then sample multiple positive examples for each example in the batch. After the pre-training stage, we use Eq. (2) to fine-tune parameters on human-labeled data to learn classifiers for FGET. In this section, we evaluate the effectiveness of our framework CROSS-C on two typical entity-related datasets: Open-Entity and Few-NERD. For each dataset, we conduct experiments in both low-resource (few-shot or zero-shot) and full-set settings. In addition to quantitative experiments, to further show how our method works, we also provide some visualization of feature spaces for qualitative analysis. Open-Entity (Choi et al., 2018) and Few-NERD (Ding et al., 2021) are both popular FGET datasets. Open-Entity includes 9 general types and 121 fine-grained types. Each example in Open-Entity may correspond to multiple entity types. Few-NERD includes 8 general types and 66 fine-grained types. Both of these two datasets have a clear type hierarchy, which is suitable for evaluating the model performance on the entity typing task. In our experiments, we require models to predict both general types and fine-grained types for each entity mention in sentences. In this paper, we select English as a high-resource language and Chinese as a low-resource language. We attempt to use human-labeled English data and large-scale unlabeled multi-lingual data for learning, to obtain an effective Chinese FGET model. This is very difficult, since no any Chinese human-labeled data is used in this process. To obtain distantly-supervised data, we apply our heuristic rules to automatically annotate the English and Chinese Wikipedia pages 2 . We then use machine translation (Klein et al., 2017; Tan et al., 2020) to translate the English distant-supervised examples into corresponding Chinese versions for cross-lingual contrastive learning. All test sets of Open-Entity and Few-NERD are translated into Chinese for evaluation. Although the test set built by machine translation may exist some errors, the overall semantics of the translated examples can still support determining the types of entity mentions. Taking human-labeled examples for evaluation is better, yet large-scale human-annotated entity typing datasets are still lacking. The experiments are performed under three settings: Few-shot setting . This setting requires models to infer entity types with a few supervised examples. We randomly sample 2, 4, 8, 16 examples for each entity type for training. Zero-shot setting . This setting requires models to infer entity types without any supervised training, i.e., no human-labeled example is used for training. Full-set setting . In this setting, all supervised examples in datasets are used for training. We follow the widely-used setting of Ling and Weld (2012), use the loose micro F 1 scores to evaluate the performance of models. model C ROSS -C . We use F-T to denote directly using English human-labeled data to fine-tune M-BERT, which is demonstrated the effectiveness in Selvaraj et al. (2021). We use M ONO C to denote only using mono-lingual contrastive learning objectives for pre-training, and then use English human-labeled data to fine-tune pre-trained parameters. All above-mentioned models are optimized by AdamW with the learning rate { 5e-6,1e-5,3e-5,5e-5 } . The batch size used for pre-training and fine-tuning is from { 8,16,32,64,128,256 } . For cross-lingual contrastive learning, we only traverse large-scale distantly-supervised data once. For fine-tuning models on human-labeled data, the epochs are from { 1,3,5,7,10 } . The temperature  used for the cosine similarity is 0 . 5 . (1) Using a multi-lingual PLM as the backbone can lead to an effective FGET model for those low-resource languages. All methods, including both the baseline models and our CROSS-C, can achieve non-trivial entity typing results on the Chinese test sets, without using any Chinese human-labeled examples for training models. (2) Using distantly-supervised data for contrastive learning can significantly improve the typing capabilities of the backbone PLMs. Compared with directly fine-tuning a multi-lingual PLM with human-labeled data in high-resource languages, conducting contrastive learning on multi-lingual distantly-supervised data can better bridge high-resource languages and low-resource languages, which is beneficial to obtain effective models in low-resource languages. (3) Compared with mono-lingual contrastive learning, our cross-lingual contrastive learning can better improve the transfer of typing knowledge from high-resource languages to low-resource languages. Our CROSS-C achieves the best results in all shot settings. And the improvements of CROSSC will gradually increase as the number of shots decreases. These results show that our method can effectively improve model performance for low-resource languages even without any high-quality supervised language-specific data. (1) In our low-resource settings, although there are no human-labeled Chinese data at all, there are still some high-quality English examples for each entity type. Therefore, the improvements of contrastive learning on the English test sets are not as obvious as on the Chinese test sets. However, compared with directly fine-tuning PLMs, contrastive learning methods still bring significant improvements, demonstrating the power of using distant supervision for data augmentation. (2) Owing to multi-lingual data, which makes models in multiple languages learn from each other, our cross-lingual contrastive learning further brings additional improvements over the mono-lingual contrastive learning. This proves the effectiveness of our cross-lingual contrastive framework. Table 3 shows the results of zero-shot entity typing on the Chinese test sets. In this table, we can see that: without a trained type classifier, our cross-lingual contrastive learning still brings the backbone PLM a strong type recognition ability in the pre-training stage. We show the model performance curve as the number of supervised examples increases in Figure 3. Note that only the supervised examples of the high-resource language English are used for training models. There is still no human-labeled data for the low-resource language Chinese. The results in the figure show that: (1) For high-resource languages, by using more supervised examples, the improvements brought by contrastive learning are gradually decreasing, which is in line with our intuition. But we should also notice that even in the full-set setting, contrastive learning methods achieve comparable or even slightly better results than fine-tuning PLMs. This means that taking contrastive learning can well reduce the impact of data noise while enhanc-2246 Figure 3: The model performance (%) curve as the number of supervised examples increases. ing performance by making full use of distantly-supervised data. (2) In both low-resource and full-set settings, the results of our contrastive learning on the Chinese test sets are always significantly higher than other baseline models. This shows that our framework can utilize the supervised data of high-resource languages and large-scale unlabeled multi-lingual data to handle FGET for low-resource languages. In order to show how our CROSS-C works more intuitively, we conduct comprehensive ablation experiments. The results of the ablation experiments are shown in Table , where   cc means that we drop the cross-lingual contrastive objective for pretraining the backbone PLM,   zc means that we drop the mono-lingual contrastive objective on the Chinese distantly-supervised data, and   ec means that we drop the mono-lingual contrastive objective on the English distantly-supervised data. From Table , we can find that: both the monolingual contrastive objectives and the cross-lingual objective play an important role in enhancing the backbone PLM, and the combination of them can lead to greater improvements. This is also the reason that our cross-lingual contrastive learning includes both mono-lingual and cross-lingual contrastive objectives for pre-training the backbone. We also give the visualization of the model during the ablation experiments of CROSS-C in Figure 4. From the visualization results, we can find that it is difficult to bridge high-resource languages and low-resource languages without using any contrastive learning. As we gradually increase the number of contrastive learning objectives, the distinction between entity types becomes more obvious, and the fusion of multi-lingual semantics also becomes better. As one of the most important tasks in the field of information extraction, FGET has been studied for a long time. Ling and Weld (2012); Yosef et al. (2012) first propose to classify named entity mentions into various fine-grained entity types, instead of just a few coarse-grained types (Sang and De Meulder, 2003; Hovy et al., 2006). Since fine-grained types bring informative semantics for language understanding, these types are widely used to enhance entity-related NLP tasks, such as coreference resolution (Khosla and Rose, 2020), entity linking (Onoe and Durrett, 2020; Chen et al., 2020a), relation extraction (Ren et al., 2017; Zhou and Chen, 2021) and event extraction (Nguyen et al., 2016; Yang and Mitchell, 2016). Some recent efforts further incorporate entity types to learn 2247 Model Open-Entity (Chinese) 2-Shot 4-Shot 8-Shot 16-Shot CROSS-C 48.1 50.1 55.5 64.6  cc 45.2  2 . Distantly-supervised FGET methods . Since entity types have complex hierarchies, manually annotating FGET data is not easy, and thus the low-resource problem is one of the key challenges of FGET. To alleviate this issue, distantly-supervised methods have been widely explored for FGET. One typical distantly-supervised approach is using knowledge bases to automatically annotate entities mentioned in the text. Ling and Weld (2012); Gillick et al. (2014) collect anchors in the Wikipedia pages that correspond to entities in knowledge bases, and then label these anchors with entity types in knowledge bases. This approach is then followed by a series of works (Ren et al., 2017; Xin et al., 2018; Choi et al., 2018; Dai et al., 2019; Jin et al., 2019; Lee et al., 2020) to obtain pseudo labels. Other approaches use various noun phrases in sentences as type pseudo labels (Dai et al., 2020, 2021), which can make full use of the recently proposed PLMs for data augmentation. Human-labeled FGET datasets . In addition to the distantly-supervised methods, the construction of FGET datasets is also advancing. CoNLL (Sang and De Meulder, 2003) and Ontonotes (Hovy et al., 2006) are the earliest datasets, although they just cover several coarse-grained types. Then, Ling and Weld (2012); Gillick et al. (2014); Ding et al. (2021) introduce about a hundred fine-grained types and annotate a large number of examples for each type. Choi et al. (2018) further extend FGET by introducing an ultra-fine set containing thousands of types. Since annotating FGET examples is time-consuming and labor-intensive, many of the ultra-fine types proposed by Choi et al. (2018) only have distantly-supervised examples. However, all these efforts only focus on English. There are also some efforts to build datasets in other languages, such as Chinese (Lee et al., 2020), Japanese (Suzuki et al., 2016), Dutch and Spanish (van Erp and Vossen, 2017), but the scale and quality of these non-English datasets are still not comparable with English datasets, i.e., non-English human-labeled data are still scarce. Cross-lingual and contrastive learning for FGET . Although cross-lingual learning has been widely explored in entity linking (Sil et al., 2018; Upadhyay et al., 2018; Rijhwani et al., 2019) and named entity recognition (Pan et al., 2017; Xie et al., 2018; Rahimi et al., 2019; Zhou et al., 2019), cross-lingual entity typing has not yet been explored much (Selvaraj et al., 2021). For contrastive learning (Chen et al., 2020b; Oord et al., 2018), some preliminary works have explored it for extracting relations between entities (Soares et al., 2019) and achieved promising results. Peng et al. (2020) further use contrastive learning to analyze the impact of entity information on relation extraction. Similar to cross-lingual learning, the exploration of contrastive learning for FGET is still in the preliminary stage. In this paper, to learn effective FGET models for those low-resource languages, we propose an effective cross-lingual contrastive learning framework CROSS-C to transfer the typing knowledge from high-resource languages to low-resource languages. Specifically, the framework CROSS-C uses a multi-lingual PLM  M-BERT as the framework backbone, which can simultaneously capture multi-lingual semantics in a unified feature space. Furthermore, to bridge the gap between high-resource languages and low-resource languages, we introduce entity-pair-oriented heuristic rules as well as machine translation to automatically obtain high-quality cross-lingual data, and then apply cross-lingual contrastive learning on this distantly-supervised data to enhance the backbone PLM. The experimental results show that by applying CROSS-C, the typing knowledge can be transferred from high-resource languages to low-resource languages, and we can learn effective FGET models without any language-specific 2248 human-labeled data for those low-resource languages. Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Hai-Tao Zheng, and Zhiyuan Liu. Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, and David Huynh. 2014. Context-dependent fine-grained entity type tagging. arXiv preprint arXiv:1412.1820 . Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. 2021. Pre-trained models: Past, present and future. AI Open , 2:225250. Eduard Hovy, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% solution. In Proceedings of NAACL-HLT , pages 5760. Sopan Khosla and Carolyn Rose. 2020. Using type information to improve entity coreference resolution. In Proceedings of the Workshop of ACL , pages 2031. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senel-lart, and Alexander M Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. In Proceedings of ACL, System Demonstrations , pages 67 72. Chin Lee, Hongliang Dai, Yangqiu Song, and Xin Li. 2020. A chinese corpus for fine-grained entity typing. In Proceedings of LREC , pages 44514457. Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2020. A survey on deep learning for named entity recognition. TKDE . Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2020. K-bert: Enabling language representation with knowledge graph. In Proceedings of AAAI , pages 29012908. Thien Huu Nguyen, Kyunghyun Cho, and Ralph Gr-ishman. 2016. Joint event extraction via recurrent neural networks. In Proceedings of NAACL-HLT , pages 300309. Yasumasa Onoe and Greg Durrett. 2020. Fine-grained entity typing for domain independent entity linking. In Proceedings of AAAI , pages 85768583. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 . Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Noth-man, Kevin Knight, and Heng Ji. 2017. Cross-lingual name tagging and linking for 282 languages. In Proceedings of ACL , pages 19461958. 2249 Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. (1) We will explore how to better utilize unsupervised data to deal with the low-resource problem of FGET, such as using better PLMs and more effective tuning methods. (2) We will also promote the construction of cross-lingual FGET datasets, which will advance the development of FGET in specific languages, especially for those low-resource languages other than English. This work is supported by the National Key R&D Program of China (No. 2020AAA0106502), Institute Guo Qiang of Tsinghua University, Beijing Academy of Artificial Intelligence (BAAI), and International Innovation Center of Tsinghua University. This work is also supported by Tencent AI Platform Department.
abstain abstain abstain objective method result result other abstain abstain abstain abstain other abstain abstain method abstain abstain abstain method abstain method result abstain method method method abstain result method method method abstain abstain abstain method method method abstain method method method method abstain method method abstain other other abstain method abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain method method method abstain abstain other abstain method abstain method abstain abstain method method abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain other other other other other other other other other other other other other other other other other other other other other other other other objective abstain result result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain objective abstain other other