transformers safetensors distilbert token-classification JNLPBA NER en base_model:distilbert/distilbert-base-uncased base_model:finetune:distilbert/distilbert-base-uncased license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
base_model:
- distilbert/distilbert-base-uncased
language:
- en
library_name: transformers
license: apache-2.0
metrics:
- seqeval
tags:
- JNLPBA
- NER
---

# Model Card for Model ID

Fine-tuned distilbert model. Trained on train set of JNLPBA dataset taken from [BLURB](https://microsoft.github.io/BLURB/tasks.html). 


## Model Details


### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/kbulutozler/medical-llm-benchmark



## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Train set of JNLPBA dataset.

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Classical fine-tuning. 

#### Training Hyperparameters

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

learning_rate=5e-5
per_device_train_batch_size=16
per_device_eval_batch_size=16
num_train_epochs=3
weight_decay=0.01


## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

Test set of JNLPBA dataset.

### Results

Precision: 0.73
Recall: 0.83
Micro-F1: 0.78


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

- **Hardware Type:** 1xRTX A4000
- **Hours used:** 00:19:00

