Few-shot NER needs to effectively capture information from limited instances and transfer useful knowledge from external resources. In this paper, we propose a self-describing mechanism for few-shot NER, which can effectively leverage illustrative instances and precisely transfer knowledge from external resources by describing both entity types and mentions using a universal concept set. Specifically, we design Self-describing Networks (SDNet), a Seq2Seq generation model which can universally describe mentions using concepts, automatically map novel entity types to concepts, and adaptively recognize entities on-demand. We pre-train SDNet with large-scale corpus, and conduct experiments on 8 benchmarks from different domains. Experiments show that SDNet achieves competitive performances on all benchmarks and achieves the new state-of-the-art on 6 benchmarks, which demonstrates its effectiveness and robustness. Few-shot named entity recognition (FS-NER) aims to identify entity mentions corresponding to new entity types (i.e., novel types) with only a few illustrative examples. FS-NER is a promising technique for open-domain NER which contains various unforeseen types and very limited examples and therefore has attached great attention in recent years (Huang et al., 2020; Wang et al., 2021). The main challenge of FS-NER is how to accurately model the semantics of unforeseen entity types using only a few illustrative examples. To achieve this, FS-NER needs to effectively capture information in few-shot examples, meanwhile exploiting and transferring useful knowledge from external resources. Unfortunately, information entailed in illustrative examples is very limited, i.e., the limited information challenge . And external  Equally Contribution. knowledge usually doesn't directly match with the new task because it may contain irrelevant, heterogeneous, or even conflicting knowledge (Beryozkin et al., 2019; Yu and Yang, 2020)  which we refer as knowledge mismatch challenge . For example, the schemas in Wikipedia, OntoNotes (Ralph et al., 2013) and WNUT17 (Derczynski et al., 2017) are conflicting, where America is geographic entity in Wikipedia, GPE in OntoNotes, and location in WNUT17. Such a knowledge mismatch problem makes it unsuitable to directly transfer external knowledge to downstream tasks. Consequently, how to sufficiently leverage limited few-shot examples and precisely transfer external knowledge are the critical challenges for FS-NER. To this end, this paper proposes a self-describing mechanism for FS-NER. The main idea behind self-describing mechanism is that all entity types can be described using the same set of concepts, and the mapping between types and concepts can be universally modeled and learned. In this way, the knowledge mismatch challenge can be resolved by uniformly describing different entity types using 5711 the same concept set. For example, in Figure 1 the types in different schemas are mapped to the same concept set { park , garden , country , ...}, therefore the knowledge in different sources can be universally described and transferred. Furthermore, because the concept mapping is universal, the few examples are only used to construct the mapping between novel types and concepts, the limited information problem can be effectively addressed. Based on the above idea, we propose Self-describing Networks  SDNet, a Seq2Seq generation network which can universally describe mentions using concepts, automatically map novel entity types to concepts, and adaptively recognize entities on-demand. Specifically, to capture the semantics of a mention, SDNet generates a set of universal concepts as its description. For example, generate { capital, city } for Dr. Kohl came to [Beijing], .... To map entity types to concepts, SDNet generates and fuses the concept description of the mentions with the same entity type. For example, map GPE to { country , capital , city } using its mentions Beijing and America. To recognize entity, SDNet directly generates all entities in a sentence via a concept-enriched prefix prompt, which contains the target entity types and their concept descriptions. For example, recognizing entity in France is beautiful. by generatingFrance is GPE using prefix prompt  [EG] GPE : { country , capital , city }. Because the concept set is universal, we pretrain SDNet on large-scale, easily accessible web resources. Concretely, we collect a pre-training dataset which contains 56M sentences with more than 31K concepts by leveraging the links from Wikipedia anchor words to the Wikidata items. By projecting both mentions and entity types to a universal concept space, SDNet can effectively enrich entity types to resolve the limited information problem, universally represent different schemas to resolve the knowledge mismatch problem, and can be effectively pre-trained in a unified way. Moreover, all the above tasks are modeled in a single generation model by using prefix prompt mechanism to distinguish different tasks, which makes the model controllable, universal and can be continuously trained. We conduct experiments on 8 few-shot NER benchmarks with different domains. Experiments show that SDNet leads to very competitive performance and achieves the new state-of-the-art on 6 of these benchmarks. 1 Generally speaking, the contributions of this paper are:  We propose a self-describing mechanism for FS-NER, which can effectively resolve the limited information challenge and the knowledge mismatch challenge by describing both entity types and mentions using a universal concept set. We propose Self-describing Networks  SDNet, a Seq2Seq generation network which can universally describe mentions using concepts, automatically map novel entity types to concepts, and adaptively recognize entities on-demand. We pre-train SDNet on the large-scale open dataset, which provides a universal knowledge for few-shot NER and can benefit many future NER studies. To deal with the limited information challenge, current FS-NER studies mostly focus on leveraging external knowledge, many knowledge resources are used: 1) PLMs. Early FS-NER studies (Tong et al., 2021; Wang et al., 2021) mainly use PLMs for better encoding. And prompt-based NER formulation is proposed to exploit the PLMs' knowledge more effectively (Xin et al., 2018; Obeidat et al., 2019; Dai et al., 2021; Ding et al., 2021; Yan et al., 2021a; Liu et al., 2021; Cui et al., 2021; Ma et al., 2021; Lee et al., 2021). 2) Existing annotation datasets. These studies (Fritzler et al., 2019; Hou et al., 2020; Yang and Katiyar, 2020; Li et al., 2020a,b; Tong et al., 2021; Das et al., 2021) focus on reusing annotations in existing datasets, and the annotations can be used to pre-train NER models. 3) Distantly annotated datasets. Some works (Mengge et al., 2020; Huang et al., 2020; Jiang et al., 2021) try to automatically construct NER datasets via distant supervision, but which often suffer from the partially-labeled (Yang et al., 2018; Nooralahzadeh et al., 2019; Peng et al., 2019) and noise label (Shang et al., 2018; Peng et al., 2019; Zhang et al., 2021b,a) problem. To deal with the knowledge mismatch problem, Kim et al. (2015); Reed et al. (2016); Qiao et al. (2016); Xian et al. (2019); Hou et al. (2020) employ label project methods which project labels 1 Our source codes are openly available at https:// github.com/chen700564/sdnet 5712 in different schemas. Rei and Sgaard (2018); Li et al. (2020c); Wang et al. (2021); Aly et al. (2021) enrich the semantics of labels using manually label descriptions. Beryozkin et al. (2019); Yu and Yang (2020) merge the labels in different schemas into the same taxonomy for knowledge sharing. And Jiang et al. (2021) relabels the external noisy datasets using current labels. Compared with these methods, we resolve the knowledge mismatch problem by mapping all entity types to a universal concept set, and the concept mapping and target entities are automatically generated using a self-describing networks. 3 Self-describing Networks for FS-NER In this section, we describe how to build few-shot entity recognizers and recognize entities using Self-describing networks. Figure 3 (b) shows the entire procedure. Specifically, SDNet is a Seq2Seq network which performs two generation tasks successively 1) Mention describing , which generates the concept descriptions of mentions; 2) Entity generation , which adaptively generates entity mentions corresponding to desirable novel type one by one. Using SDNet, NER can be directly performed through the entity generation process by putting type descriptions into its prompt. Given a novel type, its type description is built through mention describing upon its illustrative instances. In the following, we will first introduce SDNet, then describe how to construct type descriptions and build few-shot entity recognizers. SDNet is a Seq2Seq network that can perform two generation tasks: mention describing and entity generation. Mention describing is to generate the concept descriptions of mentions and entity generation is to adaptively generate entity mentions. To guide the above two processes, SDNet uses different task prompts P and generates different outputs Y . Figure 2 shows their examples. For mention describing, the prompt contains a task descriptor [MD] , and the target entity mentions. For entity recognition, the prompt contains a task descriptor [EG] , and a list of target novel types and their corresponding descriptions. Taking prompt P and sentence X as input, SDNet will generate a sequence Y which contains the mention describing or entity generation results. The above two processes can be viewed as symmetrical processes : (cid:5)(cid:13)(cid:16)(cid:21)(cid:15)(cid:17)(cid:16)(cid:1)(cid:12)(cid:13)(cid:20)(cid:11)(cid:19)(cid:15)(cid:10)(cid:15)(cid:16)(cid:14) (cid:4)(cid:16)(cid:18)(cid:22)(cid:21)(cid:1) (cid:16)(cid:21)(cid:15)(cid:21)(cid:23)(cid:1)(cid:14)(cid:13)(cid:16)(cid:13)(cid:19)(cid:9)(cid:21)(cid:15)(cid:17)(cid:16) (cid:7)(cid:22)(cid:21)(cid:18)(cid:22)(cid:21)(cid:1) {[MD] Harry Potter ; J.K. Rowling ;} Harry Potter is written by J.K. Rowling. one is to capture concept semantics of given entities, the other is to identify entities containing the specific concepts. Specifically, SDNet first concatenates prompt P and sentence X into a sequence I = P  X and then fed I into an encoder to obtain the hidden state representation H : H = Encoder( I ) . Then H will be fed into a decoder, and the decoder will sequentially generate a sequence Y . At time step t, the probability p t of generating tokens in vocabulary is calculated by: p t = Decoder( H , Y <t ) . We use the greedy decoding here and therefore the word in the target vocabulary with maximum value in p t is generated until [ EOS ] is generated. By modeling different tasks in a single model, the generation is controllable, learning is uniform, and the model can be continuously trained. We can see that, few-shot entity recognition can be effectively performed using the above two generation processes. For entity recognition, we can put the descriptions of target entity types into the prompt, then entities will be adaptively generated through the entity generation process. To construct the entity recognizer of a novel type, we only need its type description, which can be effectively built by summarizing the concept descriptions of their illustrative instances. In SDNet, entity recognition is performed by the entity generation with the given entity generation prompt PEG and sentence X . Specifically, PEG starts with a task descriptor [EG] , and the descriptor is followed by a list of target types and their corresponding descriptions, i.e., PEG = { [EG] t 1 : { l 11 , . . . , l m 1 1 } ; t 2 : { l 12 , . . . , l m 2 2 } ; . . . } , 5713 GPE: country, sovereign state,  person: politician, person, organization : enterprise, where l ji is the j-th concept of i-th type t i . Prompt PEG and sentence X will be fed to SDNet as Section 3.1 described. Then, SDNet will generate text Y in the format as  e 1 is t y 1 ; . . . ; e n is t y n . , where t y i is the type of i-th entity e i . Based on the generated text Y , the recognized entities are obtained, i.e, {< e 1 , t y 1 > ... < e n , t y n >}. We can see that, in SDNet, the entity generation process can be controlled on-the-fly using different prompts. For example, given a sentence Harry Potter is written by J.K. Rowling., if we want to identify entity of person type , put { [EG] person : { actor , writer }} to PEG , SDNet will generate J.K. Rowling is person, while if we want to identify entity of creative_work type, put { [EG] creative_work : { book , music }} to PEG , SDNet will generate Harry Potter is cre-ative_work. SDNet is controlled on-the-fly to generate different types of entities by introducing different corresponding type descriptions to PEG . For example, the description { actor , doctor , ...} for and the description is { city , state , ...} for location . To build the type description for novel types with several illustrative examples, SDNet first obtains the concept description of each mention in illustrative examples via mention describing. Then the type description of each type is constructed by summarizing all the concept descriptions of its illustrative examples. In the following, we describe them in detail. Mention Describing. In SDNet, mention describing is a generation process, whose input is mention describing prompt PMD and an illustrative instance X . Specifically, given an illustrative example X which contains entity mentions { e 1 , e 2 , . . . } of novel types, PMD starts with a task descriptor [MD] , and the descriptor is followed by target entity mentions. i.e., PMD = { [MD] e 1 ; e 2 ; . . . } . Prompt PMD and sentence X will be fed to SDNet as Section 3.1 described. And then SDNet will generate the text Y in the format as  e 1 is l 11 , . . . , l n 1 1 ; e 2 is l 12 , . . . , l n 2 2 ; . . . , where l ji is the j-th concept for the i-th entity mention. The concept set { l 1 i , l 2 i , . . . , l n i i } will be considered as the semantic concepts reflected by entity mention e i . Type Description Construction. SDNet then summarizes the generated concepts to describe the precise semantics of specific novel types. Specifically, all concept descriptions of mentions with the same type t will be fused to C and regarded as the description of type t . And the type descriptions M = { ( t, C ) } are constructed. Then the constructed type descriptions are incorporated to PEG to guide entity generation. Filtering Strategy. Because of the diversified downstream novel types, SDNet may not have suffi-cient knowledge for describing some of these types, and therefore forcing SDNet to describe them can result in the inaccurate descriptions. To resolve this problem, we introduce a filtering strategy to 5714 make SDNet able to reject generating unreliable descriptions. Specifically, SDNet is trained to generate other as the concept description for those uncertain instances. Given a novel type and a few illustrative instances, we will count the frequency of other in the concept descriptions from these instances. If the frequency of generating other on illustrative instances is greater than 0.5, we will remove the type description, and directly use the type name as PEG . We will describe how SDNet learns the filtering strategy in Section 4.1. In this section, we first describe how to pre-train SDNet using large-scale external data, so that the common NER ability can be captured through the mention describing ability and the entity generation ability. Then we describe how to quickly adapt and transfer NER knowledge via fine-tuning. Figure 3 shows the two processes and we describe them as follows. In SDNet, the NER ability consists of mention describing ability and entity generation ability, which can be effectively pre-trained by constructing corresponding datasets. This paper constructs datasets and pre-trains SDNet using the easily available and large-scale Wikipedia and Wikidata data. Entity Mention Collection. For SDNet pretraining, we need to collect < e, T, X > triples, where e is entity mention, T is entity types and X is sentence, such as <J.K. Rowling; person, writer, ...; J.K. Rowling writes ...>. To this end, we use the 20210401 version of Wikipedia and Wikidata dump and collect triples by aligning facts in Wikidata and documents in Wikipedia and process as follows. 1) Firstly, we construct an entity type dictionary from Wikidata. We regard each item in Wikidata as an entity and use the instance of, subclass of and occupation property values as its corresponding entity types. To learn general NER knowledge, we use all entity types except whose instances are < 5. For the types whose names are longer than 3 tokens, we use their head words as the final type for simplicity, e.g., state award of the Republic of Moldova is converted to state award. In this way, we obtain a collection T of 31K types which can serve as a solid foundation for universal NER. 2) Secondly, we collect the mentions of each entity using its anchor texts in Wikipedia and the top 3 frequent noun phrase occurrences of its entry page (Li et al., 2010). Then for each mention, we identify its entity types by linking it to its Wikidata item's types. If its Wikidata item doesn't have a type, we assign its type as other . For each Wikipedia page, we split the text to sentences 2 and filter out sentences that have no entities. Finally, we construct a training dataset containing 56M instances. Type Description Building. To pre-train SDNet, we need the concept descriptions MP = { ( t i , C i ) } , where t i  T , C i is the related concepts of type t i . This paper uses the collected entity types above as concepts, and builds the type description as follows. Given an entity type, we collect all its co-occurring entity types as its describing concepts. For example, Person can be described as { businessman , CEO , musician , musician ...} by collecting the types of Steve Jobs: { person , businessman , CEO } and Beethoven: { person , musician , pianist }. In this way, for each entity type we have a describing concept set. Because some entity types have a very large describing concept set, we randomly sample no more than N (10 in this paper) concepts during pre-training for efficiency. Pre-training via Mention Describing and Entity Generation. Given a sentence X with its mention-type tuples { ( e i , T i ) | e i  E, T i  T } , where T i = { t 1 i , ..., t n i i } is the set of types of i-th entity mention e i , t ji is the j-th type of the e i , E = { e 1 , e 2 , ... } is the set of entity mentions contained in X . Then we construct type descriptions, and transform these triples to pre-training instances. Specifically, for mention describing, some target entity mentions E (cid:48) are sampled from E to put into prompt PMD . Then SDNet will take PMD and X to generate the corresponding types of sampled mentions E (cid:48) as described in Section 3.3. For entity generation, positive type T p and negative type T n are sampled to construct the target-sampled type set T (cid:48) = T p  T n , where T p  T 1  T i ...  T k , T n  T \ { T 1  T i ...  T k } . Next, the type set T (cid:48) and their sampled concept description will be put into prompt PEG . Then SDNet will take prompt PEG and sentence X to generate the sequence as described in Section 3.2. For each instance, SDNet generates two kinds of sequences: (cid:102) Y pm for mention describing, and (cid:102) Y pe for 2 nltk.tokenize.punkt 5715 CoNLL WNUT Res Movie1 Movie2 Re3d I2B2 Onto AVE Baselines RoBERTa (Huang et al., 2020) 53.5 25.7 48.7 51.3 / / 36.0 57.7 / RoBERTa-DS (Huang et al., 2020)* 61.4 34.2 49.1 53.1 / / 38.5 68.8 / Proto (Huang et al., 2020) 58.4 29.5 44.1 38.0 / / 32.0 53.3 / Proto-DS (Huang et al., 2020)* 60.9 35.9 48.4 43.8 / / 36.6 57.0 / spanNER (Wang et al., 2021) 71.1 25.8 49.1 / 65.4 / / 67.3 / spanNER-DS (Wang et al., 2021)* 75.6 38.5 51.2 / 67.8 / / 71.6 / Baselines [in-house] Bert-base 58.6 23.2 47.6 52.4 66.3 57.0 47.6 61.1 51.7 T5-base 60.0 36.6 59.4 57.9 69.9 57.1 39.9 62.0 55.3 T5-base-prompt 55.4 34.2 58.4 58.7 67.1 60.7 61.8 59.8 57.0 T5-base-DS 68.2 34.9 59.7 58.4 70.8 56.0 34.1 58.8 55.1 Ours SDNet 71.4 44.1 60.7 61.3 72.6 65.4 64.3 71.0 63.8 Table 1: Micro-F1 scores on 8 datasets in 5-shot setting. entity generation. We use cross-entropy (CE) loss to train SDNet: L p = CE ( (cid:102) Y pm , Y pm ) + CE ( (cid:102) Y pe , Y pe ) (1) Note that when constructing the target generation sequence Y pe , the order of mentions depends on the order they appear in the original text. As described above, SDNet can directly recognize entities using manually designed type descriptions. But SDNet can also automatically build type descriptions using illustrative instances and be further improved by fine-tuning. Specifically, given annotated < e, T, X > instances, we first construct the descriptions of different types, next build an entity generation prompt PEG , then generate sequence (cid:102) Y fn . We fine-tune SDNet by optimizing: L f = CE ( (cid:102) Y fn , Y fn ) (2) We can see that, by fine-tuning SDNet, the entity generation process can better capture the associations between mentions and entity types. Datasets. Following previous studies, we use 8 benchmarks from different domains: 1) CoNLL2003 (Sang and Meulder, 2003) ; 2) WNUT17 (Derczynski et al., 2017); 3) Re3d (Sci-ence and Laborator, 2017); 4) MIT corpus (Liu et al., 2013a,b) includes three datasets: Res, Movie1(trivial10k13 version) and Movie2; 5) I2B2 (Stubbs and Uzuner, 2015); 6) OntoNotes5 (Ralph et al., 2013). Appendix shows detailed statistics of these datasets. Evaluation. We conduct main experiments on 5-shot setting as previous work (Huang et al., 2020; Wang et al., 2021), and also ranging the shot size from 5 to 100, as well as full shot for further analysis. For k-shot setting, we sample k instances for each entity type from training set as support set to fine-tune models. Specifically, all pre-trained models are trained 300k steps, all datasets are fine-trained 50 epochs and more hyperparameters are shown in Appendix. The performance is evaluated by micro-F1 on test set, and a predicted entity is correct if its entity type and offsets both match the golden entity. To obtain the offset of each mention, we extract entity mentions and their types from the generated sentence, and locate them in the original sentence. And if they are repeated, we match them in order, that is, the i-th same mention in the generated sentence will be matched to the i-th same utterances in the original sentence. We run 10 times for each dataset and report the average F1 score as Huang et al. (2020) and Wang et al. (2021) did. Baselines. We compare with following baselines: To evaluate the effect of pre-training for few-shot NER, we compare with baselines without NER-specific pre-training: 1) BERT-base , a traditional sequential BIO-based NER tagger (Wang et al., 2021) using pre-trained bert-base-uncased model. 2) T5-base , a generation-based NER baseline which uses the same generation format as SDNet but only using original t5-base model for generation. 3) T5-base-prompt , the prompt-extended version of T5-base which use entity types as prompt. To compare the effect of different knowledge transfer ways, we construct a distant supervision based baseline: 4) T5-base-DS , we further pre-train T5-base using the dataset collected in Section 4.1 as distantly supervised dataset. We 5716 WNUT Re3d Res Movie1 P R F P R F P R F P R F SDNet 54.78 37.08 44.06 63.67 67.22 65.39 63.99 57.88 60.74 63.54 59.30 61.33 w/o desp 48.78 39.51 43.54 62.15 65.87 63.95 62.60 57.44 59.88 62.93 59.61 61.22 w/o joint 50.68 37.46 42.96 62.99 65.01 63.97 63.15 57.23 60.01 62.71 58.64 60.60 w/o filter 53.57 35.01 42.23 63.49 66.63 65.00 63.31 57.40 60.17 62.99 59.07 60.96 Table 2: Ablation experiments. also compare with several recent few-shot NER methods: 5) RoBERTa-based few-shot classifier RoBERTa and its distantly-supervised pre-trained version RoBERTa-DS (Huang et al., 2020). 6) Prototypical network based RoBERTa model Proto and its distantly supervised pre-training version Proto-DS (Huang et al., 2020). 7) MRC model SpanNER which needs to design the description for each label and its distantly supervised pretraining version SpanNER-DS (Wang et al., 2021). Notice that these methods mostly only focus on task-specific entity types, by contrast, this paper focuses on building a general few-shot NER model which can recognize entities universally. 1) By universally modeling and pre-training NER knowledge in a generation architecture, the self-describing network can effectively handle few-shot NER. Compared with previous baselines, SDNet achieves competitive performance on all 8 datasets (new state-of-the-art on 6 datasets), and its performance is robust on different datasets. 2) Due to the limited information problem, transferring external knowledge to few-shot NER models are critical. Compared with BERT-base, T5-base, and T5-base-prompt, SDNet achieves 24%/16%/11% F1 improvements, which verified that SDNet provides a universal knowledge-enhanced foundation for NER and can adaptively transfer universal knowledge to enhance novel type recognition. 3) Due to the knowledge mismatch, it is challenging to transfer external knowledge effectively to novel downstream types. Using the same external knowledge sources, SDNet can achieve a 16% F1 improvement than T5-base-DS. We believe it is due to the noise, partially-labeled and heterogeneous problems in the external knowledge sources, and SDNet can effectively address these issues. To verify the performance of SDNet under different shot settings, we compare the performance of BERT, T5, and SDNet with k-shot samples where k ranges from 5 to 100. From Figure 4 we can see that 1) SDNet can achieve better performance under all different shot settings. Furthermore, the improvements are more significant on low shot settings, which verified the intuitions behind SDNet; 2) Generation-based models usually achieve better performance than classifier-based BERT model. We believe this is because generation-based model can more efficiently capture the semantics of types by leveraging the label utterances, and therefore can achieve much better performance, especially in low-shot settings. 3) SDNet significantly outperforms T5 on almost all datasets except Res . This shows the effectiveness of the proposed self-describing mechanism. For Res , we find that the main reason why T5 can achieve close performance 5717 to SDNet is the huge domain shifting between Res and Wikipedia. Such domain shifting makes SDNet frequently generate other for type descriptions, and therefore SDNet degrades to T5 in many cases. However, SDNet can still perform better than T5 on Res , which verifies the robustness of the proposed type description and the filtering strategy. To analyze the effectiveness of type description, multi-task modeling and type description filtering, we conduct following ablation experiments: 1) SDNet w/o desp: we directly use entity type as prompt, without the universal concept description, e.g. { [EG] person ; location ; . . . } ; 2) SDNet w/o joint: we split SDNet into two individual generation network, one for mention description, the other for entity generation, and trained them using same resources as SDNet; 3) SDNet w/o filter: we use all the generated concept descriptions with no filtering strategy. From Table 2 we can see that: 1) Type description is critical for SDNet to transfer knowledge and capture type semantics. By removing type description, the F1 of all datasets will decrease. We believe this is because 1) type description provides a common base for knowledge transferring, where all entity types are described using the same set of concepts; 2) the concept descriptions capture the semantics of entity types more accurately and precisely, which can better guide the NER process. 2) Joint learning mention describing and entity generation processes in a unified generation network is effective to capture type semantics. Compared with modeling two tasks separately, SDNet can achieve better performance. We believe this is because the two processes are symmetrical, and they can complement and promote each other. 3) Filtering strategy can effectively alleviate the transferring of mismatched knowledge. Removing the filtering strategy will undermine the performance on all 4 datasets. We believe this is because there exist some instances that can not be described based on the pre-trained SDNet knowledge. As a result, introducing filtering strategy can effectively prevent the mistaken knowledge transferring to these instances. In this section, we adapt SDNet to zero-shot setting, to investigate whether SDNet can achieve promising zero-shot performance without any illustrative instances. To this end, we conduct an experiment on WNUT by introducing manually created concepts as type descriptions based on annotation guideline, and the designed descriptions are shown in Appendix. Then we compare with the baseline without using type description, to see the effectiveness of the descriptions and whether SDNet can well-adapted to manually created descriptions. From Table 3, we can see that SDNet can benefit from manual description significantly. Compared with SDNet without description, incorporating manual description can improve zero-shot performance on the majority of types. Furthermore, SDNet with manual description on zero-shot setting can achieve comparable performance with few-shot settings in many entity types. This demonstrates that type description is an effective way for model to capture the semantic of novel types, which verifies the intuition of SDNet. Table 4 shows that by putting different types and its corresponding type descriptions to prompt, SDNet can generate different outputs according to the prompt. This verifies that SDNet can be controlled on-the-fly to generate different types of entities. In this paper, we propose Self-describing Networks, a Seq2Seq generation model which can universally describe mentions using concepts, automatically map novel entity types to concepts, and adaptively recognize entities on-demand. A large-scale SDNet model is pre-trained to provide universal knowledge for downstream NER tasks. Experiments on 8 datasets show that SDNet is effective and robust. For future work, we will extend self-describing mechanism to other NLP tasks like event extraction (Paolini et al., 2021; Lu et al., 2021) and complex NER tasks like nested (Lin et al., 2019) or discontinuous NER (Yan et al., 2021b). We thank all reviewers for their valuable comments. This work is supported by the National Key Research and Development Program of China (No. 2020AAA0106400), the National Natural Science Foundation of China under Grants no. U1936207, 62122077 and 62106251, and the Project of the Chinese Language Committee under Grant no. YB2003C002. This paper has no particular ethic consideration.
abstain objective objective method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain objective objective method other other other other other other other other other other other method method other other other other other objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain method abstain method method method method method method abstain method method abstain method method abstain abstain method method abstain abstain method abstain other method abstain abstain abstain abstain abstain other abstain method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain result abstain abstain result abstain abstain other result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result result abstain abstain abstain abstain abstain objective abstain abstain abstain other other other other abstain