card

README.md

@@ -0,0 +1,83 @@
+---
+language: 
+- english
+thumbnail: 
+tags:
+- token classification
+- 
+license: 
+datasets:
+- EMBO/sd-nlp `PANELIZATION`
+metrics:
+-
+---
+
+# sd-ner
+
+## Model description
+
+This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang). It was then fine-tuned for token classification on the SourceData [sd-nlp](https://huggingface.co/datasets/EMBO/sd-nlp) dataset with the `PANELIZATION` task to perform 'parsing' or 'segmentation' of figure legends into fragments corresponding to sub-panels.
+
+Figures are usually composite representations of results obtained with heterogenous experimental approaches and systems.  Breaking figures into  panels allows to identify more coherent descriptions of individual scientific experiments.
+
+## Intended uses & limitations
+
+#### How to use
+
+The intended use of this model is for 'parsing' figure legends into sub-fragments corresponding to individual panels as used in SourceData annotations (https://sourcedata.embo.org). 
+
+To have a quick check of the model:
+
+```python
+from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification
+example = """<s> Figure 1.Genetic screens reveal that ZBTB7A and ZBTB7B regulate ATXN1, and ZBTB7B shows more pronounced effect. (A) Western blot analysis of Atxn1 levels in cerebellar granule neurons (CGNs) after knockdown of Zbtb7a and/or Zbtb7b (n = 3).(B) Genetic interaction of wild‐type human ATXN1(30Q) with either ZBTB7A or ZBTB7B expressed in the Drosophila eyes. Co‐expression of ATXN1(30Q) with ZBTB7A or ZBTB7B severely disrupted the external Drosophila eye structure and increased ATXN1 levels. Scale bar: 100 μm. </s>"""
+tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)
+model = RobertaForTokenClassification.from_pretrained('EMBO/sd-panels')
+ner = pipeline('ner', model, tokenizer=tokenizer)
+res = ner(example)
+for r in res: print(r['word'], r['entity'])
+```
+
+#### Limitations and bias
+
+The model must be used with the `roberta-base` tokenizer.
+
+## Training data
+
+The model was trained for token classification using the [EMBO/sd-nlp `PANELIZATION`](https://huggingface.co/datasets/EMBO/sd-nlp) dataset wich includes manually annotated examples.
+
+## Training procedure
+
+The training was run on a NVIDIA DGX Station with 4XTesla V100 GPUs.
+
+Training code is available at https://github.com/source-data/soda-roberta
+
+- Command: `python -m tokcl.train PANELIZATION --num_train_epochs=6`
+- Tokenizer vocab size: 50265
+- Training data: EMBO/sd-nlp NER
+- Training with 8234 examples.
+- Evaluating on 2338 examples.
+- Training on 2 features: `O`, `B-PANEL_START`
+- Epochs: 6.0
+- `per_device_train_batch_size`: 32
+- `per_device_eval_batch_size`: 32
+- `learning_rate`: 0.0001
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+
+## Eval results
+
+Testing on 1131 examples from test set with `sklearn.metrics`:
+
+```
+              precision    recall  f1-score   support
+
+ PANEL_START       0.92      0.95      0.94      3285
+
+   micro avg       0.92      0.95      0.94      3285
+   macro avg       0.92      0.95      0.94      3285
+weighted avg       0.92      0.95      0.94      3285
+```