---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- fancy_dataset
metrics:
- accuracy
model-index:
- name: longformer-one-step
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: fancy_dataset
      type: fancy_dataset
      config: full_labels
      split: test
      args: full_labels
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8069798272958544
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-one-step

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the fancy_dataset dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5164
- B-claim: {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0}
- B-majorclaim: {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}
- B-premise: {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0}
- I-claim: {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0}
- I-majorclaim: {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0}
- I-premise: {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0}
- O: {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0}
- Accuracy: 0.8070
- Macro avg: {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0}
- Weighted avg: {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                               | B-majorclaim                                                         | B-premise                                                                                                           | I-claim                                                                                                               | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                            | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.7242          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                  | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.8028169014084507, 'recall': 0.08892355694227769, 'f1-score': 0.16011235955056177, 'support': 641.0} | {'precision': 0.43010291595197253, 'recall': 0.24589360137288552, 'f1-score': 0.31289970363437836, 'support': 4079.0} | {'precision': 0.6835106382978723, 'recall': 0.1259186673199412, 'f1-score': 0.2126603227141084, 'support': 2041.0} | {'precision': 0.7517079419299744, 'recall': 0.9221300742034046, 'f1-score': 0.8282432273493551, 'support': 11455.0} | {'precision': 0.7629536017331648, 'recall': 0.9112668463611859, 'f1-score': 0.8305409521937798, 'support': 9275.0} | 0.7285   | {'precision': 0.49015599990306213, 'recall': 0.3277332494570993, 'f1-score': 0.3349223664917405, 'support': 27909.0} | {'precision': 0.6933695141932649, 'recall': 0.7285105163209, 'f1-score': 0.6809195289383426, 'support': 27909.0}    |
| No log        | 2.0   | 82   | 0.5451          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                  | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.638235294117647, 'recall': 0.6770670826833073, 'f1-score': 0.6570779712339135, 'support': 641.0}    | {'precision': 0.5605615409729023, 'recall': 0.4209365040451091, 'f1-score': 0.48081769812377484, 'support': 4079.0}   | {'precision': 0.6609442060085837, 'recall': 0.6036256736893679, 'f1-score': 0.6309859154929578, 'support': 2041.0} | {'precision': 0.8157935644333904, 'recall': 0.916281099956351, 'f1-score': 0.8631224045063938, 'support': 11455.0}  | {'precision': 0.8817295464179737, 'recall': 0.8970350404312668, 'f1-score': 0.8893164448720005, 'support': 9275.0} | 0.7954   | {'precision': 0.5081805931357853, 'recall': 0.5021350572579146, 'f1-score': 0.5030457763184344, 'support': 27909.0}  | {'precision': 0.7727823747619976, 'recall': 0.7954064996954388, 'f1-score': 0.7813164854898953, 'support': 27909.0} |
| No log        | 3.0   | 123  | 0.5164          | {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0}   | {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0}    | {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0} | {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0} | {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0} | 0.8070   | {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0}    | {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0} |


### Framework versions

- Transformers 4.37.1
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1