readme: ---
  language: 
  - english
  thumbnail: 
  tags:
  - token classification
  license: agpl-3.0
  datasets:
  - EMBO/sd-nlp
  metrics:
  -
  ---
  
  # sd-ner
  
  ## Model description
  
  This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of English scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang). It was then fine-tuned for token classification on the SourceData [sd-nlp](https://huggingface.co/datasets/EMBO/sd-nlp) dataset with the `NER` configuration to perform Named Entity Recognition of bioentities. 
  
  
  ## Intended uses & limitations
  
  #### How to use
  
  The intended use of this model is for Named Entity Recognition of biological entities used in SourceData annotations (https://sourcedata.embo.org), including small molecules, gene products (genes and proteins), subcellular components, cell line and cell types, organ and tissues, species as well as experimental methods.
  
  To have a quick check of the model:
  
  ```python
  from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification
  example = """<s> F. Western blot of input and eluates of Upf1 domains purification in a Nmd4-HA strain. The band with the # might corresponds to a dimer of Upf1-CH, bands marked with a star correspond to residual signal with the anti-HA antibodies (Nmd4). Fragments in the eluate have a smaller size because the protein A part of the tag was removed by digestion with the TEV protease. G6PDH served as a loading control in the input samples </s>"""
  tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)
  model = RobertaForTokenClassification.from_pretrained('EMBO/sd-ner')
  ner = pipeline('ner', model, tokenizer=tokenizer)
  res = ner(example)
  for r in res:
      print(r['word'], r['entity'])
  ```
  
  #### Limitations and bias
  
  The model must be used with the `roberta-base` tokenizer.
  
  ## Training data
  
  The model was trained for token classification using the [EMBO/sd-nlp dataset](https://huggingface.co/datasets/EMBO/sd-nlp) dataset which includes manually annotated examples.
  
  ## Training procedure
  
  The training was run on an NVIDIA DGX Station with 4XTesla V100 GPUs.
  
  Training code is available at https://github.com/source-data/soda-roberta
  
  - Model fine-tuned: EMBO/bio-lm
  - Tokenizer vocab size: 50265
  - Training data: EMBO/sd-nlp
  - Dataset configuration: NER
  - Training with 48771 examples.
  - Evaluating on 13801 examples.
  - Training on 15 features: O, I-SMALL_MOLECULE, B-SMALL_MOLECULE, I-GENEPROD, B-GENEPROD, I-SUBCELLULAR, B-SUBCELLULAR, I-CELL, B-CELL, I-TISSUE, B-TISSUE, I-ORGANISM, B-ORGANISM, I-EXP_ASSAY, B-EXP_ASSAY
  - Epochs: 0.6
  - `per_device_train_batch_size`: 16
  - `per_device_eval_batch_size`: 16
  - `learning_rate`: 0.0001
  - `weight_decay`: 0.0
  - `adam_beta1`: 0.9
  - `adam_beta2`: 0.999
  - `adam_epsilon`: 1e-08
  - `max_grad_norm`: 1.0
  
  ## Eval results
  
  Testing on 7178 examples of test set with `sklearn.metrics`:
  
  ```
                  precision    recall  f1-score   support
  
            CELL       0.69      0.81      0.74      5245
       EXP_ASSAY       0.56      0.57      0.56     10067
        GENEPROD       0.77      0.89      0.82     23587
        ORGANISM       0.72      0.82      0.77      3623
  SMALL_MOLECULE       0.70      0.80      0.75      6187
     SUBCELLULAR       0.65      0.72      0.69      3700
          TISSUE       0.62      0.73      0.67      3207
  
       micro avg       0.70      0.79      0.74     55616
       macro avg       0.67      0.77      0.72     55616
    weighted avg       0.70      0.79      0.74     55616
  
  {'test_loss': 0.1830928772687912, 'test_accuracy_score': 0.9334821000160841, 'test_precision': 0.6987463009514112, 'test_recall': 0.789682825086306, 'test_f1': 0.7414366506288511, 'test_runtime': 61.0547, 'test_samples_per_second': 117.567, 'test_steps_per_second': 1.851}
  ```
  