transformers safetensors distilbert token-classification NER DistilBERT finetuned finer en base_model:distilbert/distilbert-base-uncased base_model:finetune:distilbert/distilbert-base-uncased license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
base_model:
- distilbert/distilbert-base-uncased
language:
- en
library_name: transformers
license: apache-2.0
metrics:
- f1
tags:
- NER
- DistilBERT
- finetuned
- finer
---

# Model Card for DistilBERT NER Model

## Model Details

### Model Description
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) for Named Entity Recognition (NER). It was fine-tuned on a domain-specific dataset to classify tokens into entities related to finance and compensation, as well as general non-entity tokens.

- **Developed by:** Burak Kilic
- **Model type:** Token Classification (NER)
- **Language(s):** English
- **Finetuned from model:** DistilBERT (uncased)

### Model Sources
- **Repository:** [distilbert-ner-finetuned](https://huggingface.co/sojimanatsu/finer-selected-4-labels)
- **Base Model:** [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased)

## Uses

### Direct Use
This model is intended for Named Entity Recognition tasks and can be directly used to identify entities in financial texts, such as:
B-DebtInstrumentInterestRateStatedPercentage
B-LineOfCreditFacilityMaximumBorrowingCapacity
B-DebtInstrumentBasisSpreadOnVariableRate1
B-AllocatedShareBasedCompensationExpense

### Out-of-Scope Use
This model is not suitable for tasks outside Named Entity Recognition or for domains unrelated to finance.

## Training Details

### Training Procedure
- **Dataset:** Custom annotated dataset with ~20,000 training examples.
- **Layers Fine-Tuned:** Fully connected classification layer added for NER.
- **Training Regime:** Mixed precision (fp16) with AdamW optimizer.

### Training Hyperparameters
| Parameter                     | Value                   |
|-------------------------------|-------------------------|
| Learning Rate                 | 2e-5                   |
| Warmup Ratio                  | 0.1                    |
| Batch Size                    | 128                    |
| Epochs                        | 8                      |
| Weight Decay                  | 0.01                   |
| Mixed Precision Training      | True                   |
| Evaluation Metric             | F1                     |

## Evaluation

### Testing Data
The model was evaluated on a test set of ~1,600 examples, balanced across multiple entity types.

### Results
- **Precision:** 96%
- **Recall:** 98%
- **F1 Score:** 97%
- **Accuracy:** 99.77%

## How to Get Started with the Model

```python
from transformers import pipeline

# Load the fine-tuned model
ner_pipeline = pipeline("ner", model="sojimanatsu/sojimanatsu/finer-selected-4-labels")

# Example text
text = "The bond yields 4.5% annually."
entities = ner_pipeline(text)
print(entities)
```

## Limitations
- Performance may degrade for texts outside the finance domain.
- Rare entities may have lower recognition rates.