metadata

language:
  - da
license: mit
model-index:
  - name: contract-ner-model-da
    results:
      - task:
          type: token-classification
          name: Token Classification
widget:
  - >-
    Medarbejderen ansættes til 35.000,00 kr. om måneden og arbejdsstedet er
    Supergaden 21, 2000 Frederiksberg.
inference:
  parameters:
    aggregation_strategy: first

contract-ner-model-da

This model is a fine-tuned version of xlm-roberta-base on a custom contracts dataset. It achieves the following results on the evaluation set:

Loss: 0.0100
Micro F1: 0.9074
Micro F1 No Misc: 0.9074

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 4356
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro F1 No Misc
1.1722	4.88	200	0.0612	0.0	0.0
0.0341	9.76	400	0.0209	0.0	0.0
0.0158	14.63	600	0.0109	0.7233	0.7233
0.0085	19.51	800	0.0097	0.7727	0.7727
0.0057	24.39	1000	0.0084	0.8319	0.8319
0.0041	29.27	1200	0.0085	0.8630	0.8630
0.0031	34.15	1400	0.0089	0.8261	0.8261
0.0023	39.02	1600	0.0066	0.8785	0.8785
0.0017	43.9	1800	0.0106	0.8164	0.8164
0.0012	48.78	2000	0.0092	0.8730	0.8730
0.0008	53.66	2200	0.0076	0.8868	0.8868
0.0007	58.54	2400	0.0075	0.9017	0.9017
0.0005	63.41	2600	0.0096	0.8806	0.8806
0.0004	68.29	2800	0.0094	0.8852	0.8852
0.0004	73.17	3000	0.0084	0.9126	0.9126
0.0003	78.05	3200	0.0083	0.8986	0.8986
0.0003	82.93	3400	0.0093	0.9144	0.9144
0.0003	87.8	3600	0.0088	0.9231	0.9231
0.0001	92.68	3800	0.0080	0.9280	0.9280
0.0002	97.56	4000	0.0100	0.9074	0.9074

Framework versions

Transformers 4.11.3
Pytorch 1.9.0+cu111
Datasets 1.14.0
Tokenizers 0.10.3