metadata
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- fancy_dataset
metrics:
- accuracy
model-index:
- name: longformer-one-step
results:
- task:
name: Token Classification
type: token-classification
dataset:
name: fancy_dataset
type: fancy_dataset
config: full_labels
split: test
args: full_labels
metrics:
- name: Accuracy
type: accuracy
value: 0.8069798272958544
longformer-one-step
This model is a fine-tuned version of allenai/longformer-base-4096 on the fancy_dataset dataset. It achieves the following results on the evaluation set:
- Loss: 0.5164
- B-claim: {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0}
- B-majorclaim: {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}
- B-premise: {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0}
- I-claim: {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0}
- I-majorclaim: {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0}
- I-premise: {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0}
- O: {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0}
- Accuracy: 0.8070
- Macro avg: {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0}
- Weighted avg: {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0}
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 41 | 0.7242 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.8028169014084507, 'recall': 0.08892355694227769, 'f1-score': 0.16011235955056177, 'support': 641.0} | {'precision': 0.43010291595197253, 'recall': 0.24589360137288552, 'f1-score': 0.31289970363437836, 'support': 4079.0} | {'precision': 0.6835106382978723, 'recall': 0.1259186673199412, 'f1-score': 0.2126603227141084, 'support': 2041.0} | {'precision': 0.7517079419299744, 'recall': 0.9221300742034046, 'f1-score': 0.8282432273493551, 'support': 11455.0} | {'precision': 0.7629536017331648, 'recall': 0.9112668463611859, 'f1-score': 0.8305409521937798, 'support': 9275.0} | 0.7285 | {'precision': 0.49015599990306213, 'recall': 0.3277332494570993, 'f1-score': 0.3349223664917405, 'support': 27909.0} | {'precision': 0.6933695141932649, 'recall': 0.7285105163209, 'f1-score': 0.6809195289383426, 'support': 27909.0} |
| No log | 2.0 | 82 | 0.5451 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.638235294117647, 'recall': 0.6770670826833073, 'f1-score': 0.6570779712339135, 'support': 641.0} | {'precision': 0.5605615409729023, 'recall': 0.4209365040451091, 'f1-score': 0.48081769812377484, 'support': 4079.0} | {'precision': 0.6609442060085837, 'recall': 0.6036256736893679, 'f1-score': 0.6309859154929578, 'support': 2041.0} | {'precision': 0.8157935644333904, 'recall': 0.916281099956351, 'f1-score': 0.8631224045063938, 'support': 11455.0} | {'precision': 0.8817295464179737, 'recall': 0.8970350404312668, 'f1-score': 0.8893164448720005, 'support': 9275.0} | 0.7954 | {'precision': 0.5081805931357853, 'recall': 0.5021350572579146, 'f1-score': 0.5030457763184344, 'support': 27909.0} | {'precision': 0.7727823747619976, 'recall': 0.7954064996954388, 'f1-score': 0.7813164854898953, 'support': 27909.0} |
| No log | 3.0 | 123 | 0.5164 | {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0} | {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0} | {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0} | {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0} | {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0} | 0.8070 | {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0} | {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0} |
Framework versions
- Transformers 4.37.1
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1