transformers pytorch bert token-classification PyTorch en dataset:conll2003 dataset:wnut_17 dataset:jnlpba dataset:conll2012 dataset:BTC dataset:dfki-nlp/few-nerd arxiv:2010.02405 model-index autotrain_compatible region:us
---
datasets:
- conll2003
- wnut_17
- jnlpba
- conll2012
- BTC
- dfki-nlp/few-nerd
language:
- en
pipeline_tag: false
tags:
- PyTorch
inference: false
model-index:
- name: bert-base-NER-reptile-5-datasets
  results:
  - task:
      type: named-entity-recognition
      name: few-shot-ner
    dataset:
      name: few-nerd-inter
      type: named-entity-recognition
    metrics:
    - type: f1
      value: 56.12
      name: 5 way 1~2 shot
    - type: f1
      value: 62.7
      name: 5-way 5~10-shot
    - type: f1
      value: 50.3
      name: 10-way 1~2-shot
    - type: f1
      value: 58.82
      name: 10-way 5~10-shot
---

# BERT base uncased model pre-trained on 5 NER datasets

Model was trained by _SberIDP_. The pretraining process and technical details are described [in this article](https://habr.com/ru/company/sberbank/blog/649609/).


* Task: Named Entity Recognition
* Base model: [bert-base-uncased](https://huggingface.co/bert-base-uncased)
* Training Data is 5 datasets: [CoNLL-2003](https://aclanthology.org/W03-0419.pdf), [WNUT17](http://noisy-text.github.io/2017/emerging-rare-entities.html), [JNLPBA](http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004), [CoNLL-2012 (OntoNotes)](https://aclanthology.org/W12-4501.pdf), [BTC](https://www.derczynski.com/papers/btc.pdf)
* Testing was made in Few-Shot scenario on [Few-NERD dataset](https://github.com/thunlp/Few-NERD) using the model as a backbone for [StructShot](https://arxiv.org/abs/2010.02405)



The model is pretrained for NER task using [Reptile](https://openai.com/blog/reptile/) and can be finetuned for new entities with only a small amount of samples.