transformers safetensors roberta token-classification ner named-entity-recognition en dataset:DFKI-SLT/few-nerd arxiv:2402.15343 base_model:numind/NuNER-v1.0 base_model:finetune:numind/NuNER-v1.0 license:cc-by-sa-4.0 model-index autotrain_compatible endpoints_compatible region:us
---
base_model: numind/NuNER-v1.0
datasets:
- DFKI-SLT/few-nerd
language:
- en
library_name: transformers
license: cc-by-sa-4.0
metrics:
- precision
- recall
- f1
- accuracy
pipeline_tag: token-classification
tags:
- token-classification
- ner
- named-entity-recognition
widget:
- text: Concern and scepticism surround Niger uranium mining waste storage plans.
    Towering mounds dot the desert landscape in northern Niger's Arlit region, but
    they are heaps of partially radioactive waste left from four decades of operations
    at one of the world's biggest uranium mines. An ambitious 10-year scheme costing
    $160 million is underway to secure the waste and avoid risks to health and the
    environment, but many local people are worried or sceptical. France's nuclear
    giant Areva, now called Orano, worked the area under a subsidiary, the Akouta
    Mining Company (Cominak). Cominak closed the site in 2021 after extracting 75,000
    tonnes of uranium, much of which went to fuelling the scores of nuclear reactors
    that provide the backbone of France's electricity supply. Cominak's director general
    Mahaman Sani Abdoulaye showcased the rehabilitation project to the first French
    journalists to visit the site since 2010, when seven Areva employees were kidnapped
    by jihadists.
- text: SE Michigan counties allege insulin gouging; Localities file lawsuit against
    pharmaceutical makers. Four metro Detroit counties filed federal lawsuits Wednesday
    against some of the nation's biggest pharmaceutical manufacturers and pharmacy
    benefit managers alleging illegal price fixing for insulin products. Macomb, Monroe,
    Wayne and Washtenaw counties filed the lawsuits in U.S. District Court in New
    Jersey against more than a dozen companies, including Lilly, Sanofi Aventis, Novo
    Nordisk, Express Scripts, Optum Rx and CVS Caremark, per their attorneys. "These
    are the first such lawsuits that have been filed in the state of Michigan and
    probably more to come," said attorney Melvin Butch Hollowell of the Miller Law
    Firm. He described the allegations during a news conference, saying that nationally
    "the pharmacies and manufacturers get together. They control about 90% of the
    market each, of the insulin market. They talk to each other secretly. And they
    jack up the prices through anticompetitive means. And what we've seen is over
    the past 20 years, when we talk about jacking up the prices, they jack them up
    1,500% in the last 20 years. 1,500%."
- text: Foreign governments may be spying on your smartphone notifications, senator
    says. Washington (CNN) — Foreign governments have reportedly attempted to spy
    on iPhone and Android users through the mobile app notifications they receive
    on their smartphones - and the US government has forced Apple and Google to keep
    quiet about it, according to a top US senator. Through legal demands sent to the
    tech giants, governments have allegedly tried to force Apple and Google to turn
    over sensitive information that could include the contents of a notification -
    such as previews of a text message displayed on a lock screen, or an update about
    app activity, Oregon Democratic Sen. Ron Wyden said in a new report. Wyden's report
    reflects the latest example of long-running tensions between tech companies and
    governments over law enforcement demands, which have stretched on for more than
    a decade. Governments around the world have particularly battled with tech companies
    over encryption, which provides critical protections to users and businesses while
    in some cases preventing law enforcement from pursuing investigations into messages
    sent over the internet.
- text: Tech giants ‘could severely disable UK spooks from stopping online harms’.
    Silicon Valley tech giants’ actions could “severely disable” UK spooks from preventing
    harm caused by online paedophiles and fraudsters, Suella Braverman  has suggested.
    The Conservative former home secretary named Facebook owner Meta , and Apple,
    and their use of technologies such as end-to-end encryption as a threat to attempts
    to tackle digital crimes. She claimed the choice to back these technologies without
    “safeguards” could “enable and indeed facilitate some of the worst atrocities
    that our brave men and women in law enforcement agencies deal with every day”,
    as MPs  began considering changes to investigatory powers laws. The Investigatory
    Powers (Amendment) Bill  includes measures to make it easier for agencies to examine
    and retain bulk datasets, such as publicly available online telephone records,
    and would allow intelligence agencies to use internet connection records to aid
    detection of their targets. We know that the terrorists, the serious organised
    criminals, and fraudsters, and the online paedophiles, all take advantage of the
    dark web and encrypted spaces
- text: Camargo Corrêa asks Toffoli to suspend the fine agreed with Lava Jato. The
    Camargo Corrêa group has asked Justice Dias Toffoli to suspend the R$1.4 billion
    fine it agreed to pay in its leniency agreement under Operation Car Wash. The
    company asked for an extension of the minister's decisions that benefited J&F
    and Odebrecht. Like the other companies, it claimed that it suffered undue pressure
    from members of the Federal Public Prosecutor's Office (MPF) to close the deal.
    Much of the request is based on messages exchanged between prosecutors from the
    Curitiba task force and former judge Sergio Moro - Camargo Corrêa requested full
    access to the material, seized in Operation Spoofing, which arrested the hackers
    who broke into cell phones. The dialogues, according to the group's defense, indicate
    that the executives did not freely agree to the deal, since they were the targets
    of lawsuits and pre-trial detentions.
model-index:
- name: nuner-v1_fewnerd_coarse_super
  results:
  - task:
      type: token-classification
      name: Named Entity Recognition
    dataset:
      name: FewNERD
      type: DFKI-SLT/few-nerd
      split: eval
    metrics:
    - type: f1
      value: 0.789195145498567
      name: F1
    - type: precision
      value: 0.7729689588593495
      name: Precision
    - type: recall
      value: 0.8061171805621528
      name: Recall
    - type: accuracy
      value: 0.9532904579000736
      name: Accuracy
---


# numind/NuNER-v1.0 fine-tuned on FewNERD-coarse-supervised

This is a [NuNER](https://arxiv.org/abs/2402.15343) model fine-tuned on the [FewNERD](https://huggingface.co/datasets/DFKI-SLT/few-nerd) dataset that can be used for Named Entity Recognition. NuNER model uses [RoBERTa-base](https://huggingface.co/FacebookAI/roberta-base) as the backbone encoder and it was trained on the [NuNER dataset](https://huggingface.co/datasets/numind/NuNER), which is a large and diverse dataset synthetically labeled by gpt-3.5-turbo-0301 of 1M sentences. This further pre-training phase allowed the generation of high quality token embeddings, a good starting point for fine-tuning on more specialized datasets.

## Model Details

The model was fine-tuned as a regular BERT-based model for NER task using HuggingFace Trainer class.

## Model labels

| Label        | Examples                                                                       |
|:-------------|:-------------------------------------------------------------------------------|
| art          | "The Seven Year Itch", "Time", "Imelda de ' Lambertazzi"                       |
| building     | "Boston Garden", "Henry Ford Museum", "Sheremetyevo International Airport"     |
| event        | "Russian Revolution", "Iranian Constitutional Revolution", "French Revolution" |
| location     | "Croatian", "the Republic of Croatia", "Mediterranean Basin"                   |
| organization | "IAEA", "Church 's Chicken", "Texas Chicken"                                   |
| other        | "BAR", "Amphiphysin", "N-terminal lipid"                                       |
| person       | "Ellaline Terriss", "Edmund Payne", "Hicks"                                    |
| product      | "Phantom", "100EX", "Corvettes - GT1 C6R"                                      |


## Uses

### Direct Use for Inference

```python
>>> from transformers import pipeline

>>> text = """Foreign governments may be spying on your smartphone notifications, senator says. Washington (CNN) — Foreign governments have reportedly attempted to spy on iPhone and Android users through the mobile app notifications they receive on their smartphones - and the US government has forced Apple and Google to keep quiet about it, according to a top US senator. Through legal demands sent to the tech giants, governments have allegedly tried to force Apple and Google to turn over sensitive information that could include the contents of a notification - such as previews of a text message displayed on a lock screen, or an update about app activity, Oregon Democratic Sen. Ron Wyden said in a new report. Wyden's report reflects the latest example of long-running tensions between tech companies and governments over law enforcement demands, which have stretched on for more than a decade. Governments around the world have particularly battled with tech companies over encryption, which provides critical protections to users and businesses while in some cases preventing law enforcement from pursuing investigations into messages sent over the internet."""

>>> classifier = pipeline(
    "ner",
    model="guishe/nuner-v1_fewnerd_coarse_super",
    aggregation_strategy="simple",
)
>>> classifier(text)

[{'entity_group': 'location',
  'score': 0.94416517,
  'word': ' Washington',
  'start': 82,
  'end': 92},
 {'entity_group': 'organization',
  'score': 0.9091303,
  'word': 'CNN',
  'start': 94,
  'end': 97},
 {'entity_group': 'product',
  'score': 0.89087117,
  'word': ' iPhone',
  'start': 157,
  'end': 163},
 {'entity_group': 'product',
  'score': 0.90455395,
  'word': ' Android',
  'start': 168,
  'end': 175},
 {'entity_group': 'location',
  'score': 0.6770658,
  'word': ' US',
  'start': 263,
  'end': 265},
 {'entity_group': 'organization',
  'score': 0.9525505,
  'word': ' Apple',
  'start': 288,
  'end': 293},
 {'entity_group': 'organization',
  'score': 0.9560463,
  'word': ' Google',
  'start': 298,
  'end': 304},
 {'entity_group': 'location',
  'score': 0.8810142,
  'word': ' US',
  'start': 348,
  'end': 350},
 {'entity_group': 'organization',
  'score': 0.9553381,
  'word': ' Apple',
  'start': 449,
  'end': 454},
 {'entity_group': 'organization',
  'score': 0.9610842,
  'word': ' Google',
  'start': 459,
  'end': 465},
 {'entity_group': 'location',
  'score': 0.58132076,
  'word': ' Oregon',
  'start': 649,
  'end': 655},
 {'entity_group': 'organization',
  'score': 0.8128647,
  'word': ' Democratic',
  'start': 656,
  'end': 666},
 {'entity_group': 'person',
  'score': 0.9909808,
  'word': ' Ron Wyden',
  'start': 672,
  'end': 681},
 {'entity_group': 'person',
  'score': 0.9864806,
  'word': ' Wyden',
  'start': 704,
  'end': 709}]
```


## Training Details

### Training Set Metrics
| Training set          | Min | Median  | Max |
|:----------------------|:----|:--------|:----|
| Sentence length       | 1   | 24.4945 | 267 |
| Entities per sentence | 0   | 2.5832  | 88  |


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.1498        | 1.0   | 2059 | 0.1477          | 0.7710    | 0.8013 | 0.7859 | 0.9522   |
| 0.1368        | 2.0   | 4118 | 0.1422          | 0.7797    | 0.8101 | 0.7946 | 0.9540   |
| 0.1139        | 3.0   | 6177 | 0.1433          | 0.7813    | 0.8145 | 0.7976 | 0.9547   |


### Framework versions

- Transformers 4.36.0
- Pytorch 2.0.0+cu117
- Datasets 2.18.0
- Tokenizers 0.15.2

## Citation

### BibTeX
```
@misc{bogdanov2024nuner,
      title={NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data}, 
      author={Sergei Bogdanov and Alexandre Constantin and Timothée Bernard and Benoit Crabbé and Etienne Bernard},
      year={2024},
      eprint={2402.15343},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
