Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains. Existing approaches only learn class-specific semantic features and intermediate representations from source domains. This affects generalizability to unseen target domains, resulting in suboptimal performances. To this end, we present CONTAINER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few-Shot NER. Instead of optimizing class-specific attributes, CONTAINER optimizes a generalized objective of differentiating between token categories based on their Gaussian-distributed embeddings. This effectively alleviates overfitting issues originating from training domains. Our experiments in several traditional test domains (OntoNotes, CoNLL'03, WNUT '17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that, on average, CONTAINER outperforms previous methods by 3%-13% absolute F1 points while showing consistent performance trends, even in challenging scenarios where previous approaches could not achieve appreciable performance. The source code of CONTAINER will be available at: https://github.com/ psunlpgroup/CONTaiNER . Named Entity Recognition (NER) is a fundamental NLU task that recognizes mention spans in unstructured text and categorizes them into a predefined set of entity classes. In spite of its challenging nature, recent deep-learning based approaches (Huang et al., 2015; Ma and Hovy, 2016; Lample et al., 2016; Peters et al., 2018; Devlin et al., 2018) have achieved impressive performance. As these supervised NER models require large-scale human-annotated datasets, few-shot techniques that can effectively perform NER in resource constraint settings have recently garnered a lot of attention. Few-shot learning involves learning unseen classes from very few labeled examples (Fei-Fei et al., 2006; Lake et al., 2011; Bao et al., 2020). To avoid overfitting with the limited available data, meta-learning has been introduced to focus on how to learn (Vinyals et al., 2016; Bao et al., 2020). Snell et al. (2017) proposed Prototypical Networks to learn a metric space where the examples of a specific unknown class cluster around a single prototype. Although it was primarily deployed in computer vision, Fritzler et al. (2019) and Hou et al. (2020) also used Prototypical Networks for few-shot NER. Yang and Katiyar (2020), on the other hand, proposed a supervised NER model that learns class-specific features and extends the intermediate representations to unseen domains. Additionally, they employed a Viterbi decoding variant of their model as "StructShot". Few-shot NER poses some unique challenges that make it significantly more difficult than other few-shot learning tasks. First, as a sequence labeling task, NER requires label assignment according to the concordant context as well as the dependencies within the labels (Lample et al., 2016; Yang and Katiyar, 2020). Second, in NER, tokens that do not refer to any defined set of entities are labeled as Outside ( O ). Consequently, a token that is labeled as O in training entity set may correspond to a valid target entity in test set. For prototypical networks, this challenges the notion of entity exam-6338 ples being clustered around a single prototype. As for Nearest Neighbor based methods such as Yang and Katiyar (2020), they are initially pretrained" with the objective of source class-specific supervision. As a result, the trained weights will be closely tied to the source classes and the network will project training set O-tokens so that they get clustered in embedding space. This will force the embeddings to drop a lot of useful features pertaining to its true target entity in the test set. Third, in few-shot setting, there are not enough samples from which we can select a validation set. This reduces the capability of hyperparameter tuning, which particularly affects template based methods where prompt selection is crucial for good performance (Cui et al., 2021). In fact, the absence of held-out validation set puts a lot of earlier few-shot works into question whether their strategy is truly "Few-Shot" (Perez et al., 2021). To deal with these challenges, we present a novel approach , CONTAINER that harnesses the power of contrastive learning to solve Few-Shot NER. CONTAINER tries to decrease the distance of token embeddings of similar entities while increasing it for dissimilar ones (Figure 1). This enables CONTAINER to better capture the label dependencies. Also, since CONTAINER is trained with a generalized objective, it can effectively avoid the pitfalls of O-tokens that the prior methods struggle with. Lastly, CONTAINER does not require any dataset specific prompt or hyperparameter tuning. Standard settings used in prior works (Yang and Katiyar, 2020) works well across different domains in different evaluation settings. Unlike traditional contrastive learners (Chen et al., 2020; Khosla et al., 2020) that optimize similarity objective between point embeddings, CONTAINER optimizes distributional divergence effectively modeling Gaussian Embeddings. While point embedding simply optimizes sample distances, Gaussian Embedding faces an additional constraint of maintaining class distribution through the variance estimation. Thus Gaussian Embedding explicitly models entity class distributions which not only promotes generalized feature representation but also helps in few-sample target domain adaptation. Previous works in Gaussian Embedding has also shown that mapping to a density captures representation uncertainties (Vilnis and McCallum, 2014) and expresses natural asymmetries (Qian et al., 2021) while showing better generalization requiring less data to achieve optimal performance (Bojchevski and Gnnemann, 2017). Inspired by these unique qualities of Gaussian Embedding, in this work we leverage Gaussian Embedding in contrastive learning for Few-Shot NER. A nearest neighbor classification scheme during evaluation reveals that on average, CONTAINER significantly outperforms previous SOTA approaches in a wide range of tests by up to 13% absolute F1-points. In particular, we extensively test our model in both in-domain and out-of-domain experiments as proposed in Yang and Katiyar (2020) in various datasets (CoNLL '03, OntoNotes 5.0, WNUT '17, I2B2) . We also test our model in a large dataset recently proposed for Few-Shot NER Few-NERD (Ding et al., 2021) where CONTAINER outperforms all other SOTA approaches setting a new benchmark result in the leaderboard. In summary, our contributions are as follows: (1) We propose a novel Few-Shot NER approach CONTAINER that leverages contrastive learning to infer distributional distance of their Gaussian Embeddings. To the best of our knowledge we are the first to leverage Gaussian Embedding in contrastive learning for Named Entity Recognition. (2) We demonstrate that CONTAINER representations are better suited for adaptation to unseen novel classes, even with a low number of support samples. (3) We extensively test CONTAINER in a wide range of experiments using several datasets and evaluation schemes. In almost every case, our model largely outperforms present SOTAs establishing new benchmark results. Given a sequence of n tokens { x 1 , x 2 , . . . x n } NER aims to assign each token x i to its corresponding tag label y i . Few-shot Setting For Few-shot NER, a model is trained in a source domain with a tag-set { C s ( i ) } and tested in a data-scarce target domain with a tag-set { C d ( j ) } where i, j are index of different tags. Since { C s ( i ) }  { C d ( j ) } =  , it is very challenging for models to generalize to unseen test tags. In an N-way K-shot setting, there are N tags in the target domain |{ C d ( j ) }| = N , and each tag is associated with a support set with K examples. Tagging Scheme For fair comparison of CONTAINER against previous SOTA models, we follow an IO tagging scheme where I-type repre-6339 Figure 2: Illustration of our proposed CONTAINER framework based on Contrastive Learning over Gaussian Embedddings: Evaluation Scheme To compare with SOTA models in Few-NERD leaderboard (Ding et al., 2021), we adpot episode evaluation as done by the authors. Here, a model is assessed by calculating the micro-F1 score over multiple number of test episodes. Each episode consists of a K-shot support set and a K-shot unlabeled query (test) set to make predictions . While Few-NERD is explicitly designed for episode evaluation, traditional NER datasets (e.g., OntoNotes, CoNLL'03, WNUT '17, GUM) have their distinctive tag-set distributions. Thus, sampling test episodes from the actual test data perturbs the true distribution that may not represent the actual performance. Consequently, Yang and Katiyar (2020) proposed to sample multiple support sets from the original development set and use them for prediction in the original test set. We also use this evaluation strategy for these traditional NER datasets. CONTAINER utilizes contrastive learning to optimize distributional divergence between different token entity representations. Instead of focusing on label specific attributes, this contradistinction explicitly trains the model to distinguish between different categories of tokens. Furthermore, modeling Gaussian Embedding instead of traditional point representation effectively lets CONTAINER model the entity class distribution, which incites generalized representation of tokens. Finally, it lets us carefully finetune our model even with a small number of samples without overfitting which is imperative for domain adaptation. As demonstrated in Figure 2, we first train our model in source domains. Next, we finetune model representations using few-sample support sets to adapt it to target domains. The training and finetuning of CONTAINER is illustrated in Algorithm 1. Finally, we use an instance level nearest neighbor classifier for inference in test sets. Figure 2 shows the key components of our model. To generate contextualized representation of sentence tokens, CONTAINER incorporates a pretrained language model encoder PLM . For proper comparison against existing approaches, we use BERT (Devlin et al., 2018) as our PLM encoder. Thus given a sequence of n tokens [ x 1 , x 2 , . . . , x n ] , we take the final hidden layer output of the PLM as the intermediate representations h i  R l  . These intermediate representations are then channeled through simple projection layer for generating the embedding. Unlike SimCLR (Chen et al., 2020) that uses projected point embedding for contrastive learning, we assume that token embeddings 6340 follow Gaussian distributions. Specifically, we employ projection network f  and f  for producing Gaussian distribution parameters:  i = f  ( h i ) ,  i = ELU ( f  ( h i ))+(1+  ) (2) where  i  R l ,  i  R l  l represents mean and diagonal covariance (with nonzero elements only along the diagonal of the matrix) of the Gaussian Embedding respectively; f  and f  are implemented as ReLU followed by single layer networks; ELU for exponential linear unit; and   e  14 for numerical stability. For calculating the contrastive loss, we consider the KL-divergence between all valid token pairs in the sampled batch. Two tokens x p and x q are considered as positive examples if they have the same label y p = y q . Given their Gaussian Embeddings N (  p ,  p ) and N (  q ,  q ) , we can calculate their KL-divergence as following: DKL [ N q ||N p ] = DKL [ N (  q ,  q ) ||N (  p ,  p )] = 1 2 (cid:18) Tr(   1 p  q ) + (  p   q ) T   1 p (  p   q )  l + log |  p | |  q | (cid:19) (3) Both directions of the KL-divergence are calculated since it is not symmetric. We first train our model in resource rich source domain having training data X tr . At each training step, we randomly sample a batch of sequences (without replacement) X  X tr from the training set having batch size of b . For each ( x i , y i )  X , we obtain its Gaussian Embedding N (  i ,  i ) by channeling the corresponding token sequence through the model (Algorithm 1: Line 3-6). We find in-batch positive samples X p for sample p and subsequently calculate the Gaussian embedding loss of x p with respect to that of all other valid tokens in the batch: X p = { ( x q , y q )  X | y p = y q , p  = q } (5)  ( p ) =  log (cid:80) ( x q ,y q ) X p exp(  d ( p, q )) / |X p | (cid:80) ( x q ,y q ) X ,p  = q exp(  d ( p, q )) (6) In this way we can calculate the distributional divergence of all the token pairs in the batch (Algorithm 1: Line 7-10 ). We do not scale the contrastive loss by any normalization factor as proposed by Chen et al. (2020) since we did not find it to be beneficial for optimization. After training in source domains, we finetune our model using a small number of target domain support samples following a similar procedure as in the training stage. As we have only a few samples for finetuning, we take them in a single batch. When multiple few-shot samples (e.g., 5-shot) are available for the target classes, the model can effectively adapt to the new domain by optimizing KL-divergence of Gaussian Embeddings as in Eq. 4. In contrast, for 1-shot case, it turns out challenging for models to adapt to the target class distribution. If the model has no prior knowledge about target classes (either from direct training or indirectly from source domain training where the target class entities are marked as O-type ), a single example might not be sufficient to deduce the variance of the target class distribution. Thus, for 1-shot scenario, we optimize d  ( p, q ) = ||  p   q || 22 , the squared euclidean distance between mean of the embedding distributions. When the model has direct/indirect prior knowledge about the target classes involved, we still optimize the KL-divergence of the distributions similar to the 5-shot scenario. We demonstrate in Table 7 that optimizing with squared euclidean distance gives us slightly better performance in 1-shot scenario. Nevertheless, in all cases with 5-shot support set, optimizing the KL-divergence between the Gaussian Embeddings gives us the best result. Early Stopping Finetuning with a small support set runs the risk of overfitting and without access to a held out validation set due to data scarcity in the target domain, we cannot keep tabs on the saturation point where we need to stop finetuning. To alleviate this, we rely on the calculated contrastive loss and use it as our early stopping criteria with a patience of 1. (Algorithm 1: Line 16-17, 24 ) 6341 Algorithm 1 Training and Finetuning of CONTAINER Require: Training data X tr , Support Data X sup , Train loss function d tr , Finetune loss function d ft , f  , f  , PLM 1: // training in source domain 2: for sampled (w/o replacement) minibatch X  X tr do 3: for all i  ( x i , y i )  X do 4:  i = f  ( PLM ( x i )) //[Eq. 1] 5:  i = ELU ( f  ( PLM ( x i ))) + (1 +  ) //[Eq. 2] 6: end for 7: for all i  ( x i , y i )  X do 8: Calculate  ( i ) as in Eq. After training and finetuning the network with train and support data respectively, we extract the pretrained language model encoder PLM for inference. Similar to SimCLR (Chen et al., 2020), we found that representations before the projection layers actually contain more information than the final output representation which contributes to better performance, so f  and f  projection heads are not used for inference. We thus calculate the representations of the test data from PLM and find nearest neighbor support set representation for inference (Wang et al., 2019; Yang and Katiyar, 2020). The PLM representations h sup j of each of the support token ( x sup j , y sup j )  X sup can be calculated as in Eq. 1. Similarly for test data X test , we get the PLM representations h test i where x test i  X test . Here we assign x test i the same label as the support token that is nearest in the PLM representation space: y test i = arg min y sup k where ( x sup k ,y sup k ) X sup || h test i  h sup k || 22 (7) Dataset Domain # Class # Sent OntoNotes General 18 76K I2B2'14 Medical 23 140K CoNLL'03 News 4 20K WNUT'17 Social 6 5K GUM Mixed 11 3.5K FEW-NERD Wikipedia 66 188K Table 1: Summary Statistics of Datasets Viterbi Decoding Most previous works (Hou et al., 2020; Yang and Katiyar, 2020; Ding et al., 2021) noticed a performance improvement by using CRFs (Lafferty et al., 2001) which removes false predictions to improve performance. Thus we also employ Viterbi decoding in the inference stage with an abstract transition distribution as in StructShot (Yang and Katiyar, 2020). For the transition probabilities , the transition between three abstract tags O , I , and I-other is estimated by counting their occurrences in the training set. Then for the target domain tag-set, these transition probabilities are evenly distributed into corresponding target distributions. The emission probabilities are calculated from Nearest Neighbor Inference stage. Comparing domain transfer results (Table 3) against other tasks (Table 2,4,5) we find that, interestingly, if there is no significant domain shift involved in the test data, contrastive learning allows CONTAINER to automatically extract label dependencies, obviating the requirement of extra Viterbi decoding stage. Datasets For evaluation, we use datasets across different domains: General (OntoNotes 5.0 (Weischedel et al., 2013)), Medical (I2B2 (Stubbs and Uzuner, 2015)), News (CoNLL'03 (Sang and De Meulder, 2003)), Social (WNUT'17 (Derczyn-ski et al., 2017)). We also test on GUM (Zeldes, 2017) that represents wide variety of texts: interviews, news articles, instrumental texts, and travel guides. The miscellany of domains makes it a challenging dataset to work on. Ding et al. (2021) argue that the distribution of these datasets may not be suitable for proper representation of Few-Shot capability. Thus, they proposed a new large scale dataset Few-NERD that contains 66 fine-grained entities across 8 coarse grained entities, significantly richer than previous datasets. A summary of these datasets is given in Table 1. Baselines We compare the performance of CONTAINER with state-of-the-art Few-Shot NER models on different datasets across several settings. We first measure the model performance in traditional NER datasets in tag-set extension and domain transfer tasks as proposed in Yang and Katiyar (2020). We then evaluate our model in Few-NERD (Ding et al., 2021) dataset that is explicitly designed for Few-Shot NER, and compare it against the Few-NERD leaderboard baselines. Similar to Ding et al. (2021), we take Prototypical Network based ProtoBERT (Snell et al., 2017; Fritzler et al., 2019; Hou et al., 2020), nearest neighbor based metric method NNShot that leverages the locality of in-class samples in embedding space, and additional Viterbi decoding based Structshot (Yang and Katiyar, 2020) as the main SOTA baselines. A common use-case of Few-Shot NER is that new entity types may appear in the same existing text domain. Thus (Yang and Katiyar, 2020) proposed to experiment tag-set extension capability using OntoNotes (Weischedel et al., 2013) dataset. The eighteen existing entity classes are split in three groups: A, B, and C, each having six classes. Models are tested in each of these groups having few sample support set while being trained in the remaining two groups. During training, all test group entities are replaced with O -tag. Since the source and destination domains are the same, the training phase will induce some indirect information about unseen target entities. So, during finetuning of CONTAINER, we optimize the KL-divergence between ouptut embeddings as in Eq. 4. We use the same entity class splits as used by Yang and Katiyar (2020) and used bert-base-cased as the backbone encoder for all models. Since they could not share the sampled support set for licensing reasons, we sampled five sets of support samples for each group and averaged the results, as done by the authors. We show these results in Table 2. We see that in different entity groups, CONTAINER outperforms present SOTAs by upto 12.75 absolute F1 points, a substantial improvement in performance. In this experiment a model trained on a source domain is deployed to a previously unseen novel text domain. Here we take OntoNotes (General) as our source text domain, and evaluate the Few-Shot performance in I2B2 (Medical), CoNLL (News), WNUT (Social) domains as in (Yang and Katiyar, 2020). We also evaluate the performance in GUM (Zeldes, 2017) dataset due to its particularly challenging nature. We show these results in Table 3. While all the other domains have almost no intersection with OntoNotes, target entities in CoNLL are fully contained within OntoNotes entities, that makes it comparable to supervised learning. For few-shot setting, Ding et al. (2021) proposed two different settings: Few-NERD (IN-TRA) and Few-NERD (INTER) . In Few-NERD 6343 Model 5-way 10-way Avg. (INTRA) train, dev, and test sets are divided according to coarse-grained types. As a result, fine-grained entity types belonging to People, Art, Product, MISC coarse grained types are put in the train set, Event, Building coarse grained types in dev set, and ORG, LOC in test set. So, there is no overlap between train, dev, test set classes in terms of coarse grained types. On the other hand, in Few-NERD (INTER) coarse grained types are shared, although all the fine grained types are mutually disjoint. Because of the restrictions of sharing coarse-grained types, Few-NERD (IN-TRA) is more challenging. Since, few-shot performance of any model relies on the sampled support set, the authors also released train, dev, test split for both Few-NERD (INTRA) and Few-NERD (INTER) . We evaluate our model performance using these provided dataset splits and compare the performance in Few-NERD leaderboard. All models use bert-base-uncased as the backbone encoder. As shown in Table 4 and Table 5, CONTAINER establishes new benchmark results in the leaderboard in both of these tests. We prudently analyze different components of our model and justify the design choices made in the scheming of CONTAINER. We also examine the results discussed in Section 4 that gives some intuitions about few-shot NER in general. Table 2-5 demonstrates that overall, in every scenario CONTAINER convincingly outperforms all other baseline approaches. This improvement is particularly noticeable in challenging scenarios, where all other baseline approaches perform poorly. For example, FEW-NERD (intra) (Table 4) is a challenging scenario where the coarse grained entity types corresponding to train and test sets do not overlap. As a result, other baseline approaches face a substantial performance hit, whereas CONTAINER still performs well. In tag-set extension (Table 2), we see a similar performance trend CONTAINER performs consistently well across the board. Likewise, in domain transfer to a very challenging unseen text domain like GUM (Zeldes, 2017), baseline models performs miserably; yet CONTAINER manages to perform consistently outperforming SOTA models by a significant margin. Analyzing these results more closely, we notice that while CONTAINER surpasses other baselines in almost every tests, more prominently in 5-shot cases. Evidently, CONTAINER is able to make better use of multiple few-shot samples thanks to distribution modeling via contrastive Gaussian Embedding optimization. In this context, note that StructShot actually got marginally higher F1-score in 1-shot CoNLL domain adaptation and 1  2 shot FEW-NERD (INTER) cases. In CoNLL, the target classes are subsets of training classes, so supervised learning based feature extractors are expected to get an advantage in prediction. On the other hand, Ding et al. (2021) carefully tuned the hyperparameters for baselines like StructShot for best performance. We could also improve performance in a similar manner, however for uniformity of model across different few-shot settings, we use the same model architecture in every test. Nevertheless, CONTAINER shows comparable performance even in these cases while significantly outperforming in every other test. Traditional contrastive learners usually optimize cosine similarity of point embeddings (Chen et al., 2020). While this has proven to work well in image data, in more challenging NLU tasks like Few-Shot NER, it gives subpar performance. We compare the performance of point embeddings with euclidean distance and cosine similarity to that of CONTAINER using Gaussian Embedding and KL-divergence in OntoNotes tag-set extension. We report these performance in Table 8 in Appendix. Basically, Gaussian Embedding leads to learning generalized representation during training, which is more suitable for finetuning to few sample target domain. In Appendix C, we examine this aspect by comparing the t-SNE representations from point 6344 embedding and Gaussian Embedding. Being a contrastive learner, CONTAINER can take advantage of extremely small support set to refine its representations through fine-tuning. To closely examine the effects of fine-tuning, we conduct a case study with OntoNotes tag-extension task using PERSON, DATE, MONEY, LOC, FAC, PRODUCT target entities. As shown in Table 6, we see that finetuning indeed improves few-shot performance. Besides, the effect of finetuning is even more marked in 5-shot prediction indicating that CONTAINER finetuning process can make the best use of few-samples available in target domain. Analyzing the results, we observe that domain transfer (Table 3) sees some good gains in performance from using Viterbi decoding. In contrast, tag-set extension (Table 2) and FEW-NERD (Table 4,5) gets almost no improvement from using Viterbi decoding. This indicates an interesting property of CONTAINER. During domain transfer the text domains have no overlap in train and test set. So, an extra Viterbi decoding actually provides additional information regarding the label dependencies, giving us some nice improvement. Otherwise, the train and target domain have substantial overlap in both tagset extension and FEW-NERD. Thus the model can indirectly learn the label dependencies through in-batch contrastive learning. Consequently, unless there is a marked shift in the target text domain, we can achieve the best performance even without employing additional Viterbi decoding. Meta Learning The idea of Few-shot learning was popularized in computer vision through Matching Networks (Vinyals et al., 2016). Subsequently, Prototypical Network (Snell et al., 2017) was proposed where class prototypical representations were learned. Test samples are given labels according to the nearest prototype. Later this technique was proven successful in other domains as well. Wang et al. (2019), on the other hand found simple feature transformations to be quite effective in few shot image recognition These metric learning based approaches have also been deployed in different NLP tasks (Geng et al., 2019; Bao et al., 2020; Han et al., 2018; Fritzler et al., 2019). Contrastive Learning Early progress was made by contrasting positive against negative samples (Hadsell et al., 2006; Dosovitskiy et al., 2014; Wu et al., 2018). Chen et al. (2020) proposed SimCLR by refining the idea of contrastive learning with the help of modern image augmentation techniques to learn robust sets of features. Khosla et al. (2020) leveraged this to boost supervised learning performance as well. In-batch negative sampling has also been explored for learning representation (Doer-sch and Zisserman, 2017; Ye et al., 2019). Storing instance class representation vectors is another popular direction (Wu et al., 2018; Zhuang et al., 2019; Misra and Maaten, 2020). Gaussian Embedding Vilnis and McCallum (2014) first explored the idea of learning word embeddings as Gaussian Distributions. Although the authors used RANK-SVM based learning objective instead of modern deep contextual modeling, they found that embedding densities in a Gaussian space enables natural represenation of uncertainty through variances. Later, Bojchevski and Gnne-mann (2017) leveraged Gaussian Embedding in Graph representation. Besides state-of-the-art performance, they found Gaussian Embedding to be surprisingly effective in inductive learning, generalizing to unseen nodes with few training data. Moreover, KL-divergence between Gaussian Embeddings allows explicit consideration of asymmetric distance which better represents inclusion, similarity or entailment (Qian et al., 2021) and preserve the hierarchical structures among words (Athiwaratkun and Wilson, 2018). Few-Shot NER Established few-shot learning approaches have also been applied in Named Entity Recognition. Fritzler et al. (2019) leveraged prototypical network (Snell et al., 2017) for few shot NER. Inspired by the potency of simple feature extractors and nearest neighbor inference (Wang et al., 2019; Wiseman and Stratos, 2019) in few-Shot learning, Yang and Katiyar (2020) used super-6345 vised learner based feature extractors for Few-Shot NER. Pairing it with abstract transition tag Viterbi decoding, they achieved current SOTA result in Few-Shot NER tasks. Huang et al. (2020) proposed noisy supervised pre-training for Few-Shot NER. However, this method requires access to a large scale noisy NER dataset such as WiNER (Ghaddar and Langlais, 2017) for the supervised pretraining. Acknowledging the shortcomings and evaluation scheme disparity in Few-Shot NER, Ding et al. (2021) proposed a large scale dataset specifically designed for this task. Wang et al. (2021) explored model distillation for Few-Shot NER. However, this requires access to a large unlabelled dataset for good performance. Very recently, prompt based techniques have also surfaced in this domain (Cui et al., 2021). However, the performance of these methods rely heavily on the chosen prompt. As denoted by the author, the performance delta can be massive (upto 19% absolute F1 points) depending on the prompt. Thus, in the absence of a large validation set, their applicability becomes limited in true few-shot learning (Perez et al., 2021). We propose a contrastive learning based framework CONTAINER that models Gaussian embedding and optimizes inter token distribution distance. This generalized objective helps us model a class agnostic feature extractor that avoids the pitfalls of prior Few-Shot NER methods. CONTAINER can also take advantage of few-sample support data to adapt to new target domains. Extensive evaluations in multiple traditional and recent few-shot NER datasets reveal that, CONTAINER consistently outperforms prior SOTAs, even in challenging scenarios. While we investigate the efficacy of distribution optimization based contrastive learning in Few-Shot NER, it will be of particular interest to investigate its potency in other domains as well. We thank the ACL Rolling Review reviewers for their helpful feedback. We also want to thank Nan Zhang, Ranran Haoran Zhang, and Chandan Akiti for their insightful comments on the paper. With CONTAINER, we have achieved state-of-the-art Few-Shot NER performance leveraging Gaussian Embedding based contrastive learning. However, the overall performance is still quite low compared to supervised NER that takes advantage of the full training dataset. Consequently, it is still not ready for deployment in high-stake domains (e.g. Medical Domain, I2B2 dataset), leaving a lot of room for improvement in future research.
abstain abstain abstain objective abstain abstain objective other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective objective method objective abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain method method method abstain method method abstain method method abstain method method method abstain method method method method method method method method abstain abstain abstain abstain method method method method method method other method method method abstain abstain method method method abstain abstain abstain method method method abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain abstain method abstain method method method method abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain result abstain abstain abstain abstain result abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain result other other other other other other other other other other other other other other other other other other other other other other other other other other other other objective objective abstain abstain objective other other abstain abstain abstain