metadata

license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - fancy_dataset
metrics:
  - accuracy
model-index:
  - name: longformer-simple
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: fancy_dataset
          type: fancy_dataset
          config: simple
          split: test
          args: simple
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8226736894908453

longformer-simple

This model is a fine-tuned version of allenai/longformer-base-4096 on the fancy_dataset dataset. It achieves the following results on the evaluation set:

Loss: 0.4587
Claim: {'precision': 0.5772692208794035, 'recall': 0.5279868297271872, 'f1-score': 0.5515292961552635, 'support': 4252.0}
Majorclaim: {'precision': 0.6656682890303257, 'recall': 0.8148487626031164, 'f1-score': 0.7327426334226252, 'support': 2182.0}
O: {'precision': 0.9301160937855679, 'recall': 0.8810781671159029, 'f1-score': 0.9049332816566081, 'support': 9275.0}
Premise: {'precision': 0.8568813181564913, 'recall': 0.8823770491803279, 'f1-score': 0.8694423131284578, 'support': 12200.0}
Accuracy: 0.8227
Macro avg: {'precision': 0.757483730462947, 'recall': 0.7765727021566337, 'f1-score': 0.7646618810907386, 'support': 27909.0}
Weighted avg: {'precision': 0.8236703495364839, 'recall': 0.8226736894908453, 'f1-score': 0.8221147085496641, 'support': 27909.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Claim	Majorclaim	O	Premise	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.5844	{'precision': 0.4909433962264151, 'recall': 0.30597365945437444, 'f1-score': 0.37699217618081715, 'support': 4252.0}	{'precision': 0.5969423210562891, 'recall': 0.3936755270394134, 'f1-score': 0.4744545705606187, 'support': 2182.0}	{'precision': 0.825323567773653, 'recall': 0.886900269541779, 'f1-score': 0.8550046772684753, 'support': 9275.0}	{'precision': 0.8012704829278856, 'recall': 0.9098360655737705, 'f1-score': 0.8521091620926572, 'support': 12200.0}	0.7699	{'precision': 0.6786199419960607, 'recall': 0.6240963804023343, 'f1-score': 0.639640146525642, 'support': 27909.0}	{'precision': 0.7460100844931877, 'recall': 0.7698591852090724, 'f1-score': 0.7511602266394221, 'support': 27909.0}
No log	2.0	82	0.4763	{'precision': 0.5736620565243535, 'recall': 0.4487300094073377, 'f1-score': 0.5035629453681709, 'support': 4252.0}	{'precision': 0.7095454545454546, 'recall': 0.7153987167736022, 'f1-score': 0.7124600638977636, 'support': 2182.0}	{'precision': 0.8935006435006435, 'recall': 0.8982210242587602, 'f1-score': 0.8958546158395613, 'support': 9275.0}	{'precision': 0.8362814916915537, 'recall': 0.8951639344262295, 'f1-score': 0.8647214854111407, 'support': 12200.0}	0.8141	{'precision': 0.7532474115655013, 'recall': 0.7393784212164823, 'f1-score': 0.7441497776291592, 'support': 27909.0}	{'precision': 0.8053779036606528, 'recall': 0.8141101436812498, 'f1-score': 0.80814042735527, 'support': 27909.0}
No log	3.0	123	0.4587	{'precision': 0.5772692208794035, 'recall': 0.5279868297271872, 'f1-score': 0.5515292961552635, 'support': 4252.0}	{'precision': 0.6656682890303257, 'recall': 0.8148487626031164, 'f1-score': 0.7327426334226252, 'support': 2182.0}	{'precision': 0.9301160937855679, 'recall': 0.8810781671159029, 'f1-score': 0.9049332816566081, 'support': 9275.0}	{'precision': 0.8568813181564913, 'recall': 0.8823770491803279, 'f1-score': 0.8694423131284578, 'support': 12200.0}	0.8227	{'precision': 0.757483730462947, 'recall': 0.7765727021566337, 'f1-score': 0.7646618810907386, 'support': 27909.0}	{'precision': 0.8236703495364839, 'recall': 0.8226736894908453, 'f1-score': 0.8221147085496641, 'support': 27909.0}

Framework versions

Transformers 4.37.1
Pytorch 2.1.2+cu121
Datasets 2.16.1
Tokenizers 0.15.1