meta_data/README_s42_e14.md

---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-simple
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: simple
      split: train[80%:100%]
      args: simple
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8449618016582787
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-simple

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6109
- Claim: {'precision': 0.6048215551878988, 'recall': 0.6139635316698656, 'f1-score': 0.6093582569353494, 'support': 4168.0}
- Majorclaim: {'precision': 0.7849462365591398, 'recall': 0.8141263940520446, 'f1-score': 0.7992700729927007, 'support': 2152.0}
- O: {'precision': 0.931049822064057, 'recall': 0.9074355083459787, 'f1-score': 0.9190910088923043, 'support': 9226.0}
- Premise: {'precision': 0.8758632028937849, 'recall': 0.8824650045556199, 'f1-score': 0.8791517101951561, 'support': 12073.0}
- Accuracy: 0.8450
- Macro avg: {'precision': 0.7991702041762201, 'recall': 0.8044976096558771, 'f1-score': 0.8017177622538776, 'support': 27619.0}
- Weighted avg: {'precision': 0.8463109688981529, 'recall': 0.8449618016582787, 'f1-score': 0.8455543885446015, 'support': 27619.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 14

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                  | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.5692          | {'precision': 0.4959266802443992, 'recall': 0.2336852207293666, 'f1-score': 0.3176777560339204, 'support': 4168.0}  | {'precision': 0.5318267419962335, 'recall': 0.6561338289962825, 'f1-score': 0.5874765966299147, 'support': 2152.0} | {'precision': 0.9195207551736657, 'recall': 0.8235421634511164, 'f1-score': 0.8688890159528846, 'support': 9226.0} | {'precision': 0.7774309560968989, 'recall': 0.9489770562411993, 'f1-score': 0.8546810891458411, 'support': 12073.0} | 0.7763   | {'precision': 0.6811762833777993, 'recall': 0.6655845673544912, 'f1-score': 0.6571811144406402, 'support': 27619.0} | {'precision': 0.7632765839539684, 'recall': 0.7763134074369094, 'f1-score': 0.7575678110552883, 'support': 27619.0} |
| No log        | 2.0   | 82   | 0.4421          | {'precision': 0.6073619631901841, 'recall': 0.45129558541266795, 'f1-score': 0.5178251892635926, 'support': 4168.0} | {'precision': 0.7044500419815282, 'recall': 0.7797397769516728, 'f1-score': 0.7401852668725187, 'support': 2152.0} | {'precision': 0.9249115599680475, 'recall': 0.8784955560372859, 'f1-score': 0.9011062315859693, 'support': 9226.0} | {'precision': 0.8318756073858115, 'recall': 0.9217261658245672, 'f1-score': 0.874499017681729, 'support': 12073.0}  | 0.8252   | {'precision': 0.7671497931313928, 'recall': 0.7578142710565485, 'f1-score': 0.7584039263509523, 'support': 27619.0} | {'precision': 0.8191436841723105, 'recall': 0.8252290090155328, 'f1-score': 0.8190957969602078, 'support': 27619.0} |
| No log        | 3.0   | 123  | 0.4286          | {'precision': 0.5590650663297536, 'recall': 0.636996161228407, 'f1-score': 0.5954917573174835, 'support': 4168.0}   | {'precision': 0.7408256880733946, 'recall': 0.7504646840148699, 'f1-score': 0.7456140350877194, 'support': 2152.0} | {'precision': 0.949893137022085, 'recall': 0.8671146759158899, 'f1-score': 0.9066183136899364, 'support': 9226.0}  | {'precision': 0.8706390609716336, 'recall': 0.8847013998177752, 'f1-score': 0.877613902469085, 'support': 12073.0}  | 0.8310   | {'precision': 0.7801057380992167, 'recall': 0.7848192302442354, 'f1-score': 0.7813345021410562, 'support': 27619.0} | {'precision': 0.839978983398119, 'recall': 0.8309859154929577, 'f1-score': 0.834442385843827, 'support': 27619.0}   |
| No log        | 4.0   | 164  | 0.4205          | {'precision': 0.6486063263388663, 'recall': 0.4968809980806142, 'f1-score': 0.5626952859665806, 'support': 4168.0}  | {'precision': 0.7857142857142857, 'recall': 0.7973977695167286, 'f1-score': 0.7915129151291513, 'support': 2152.0} | {'precision': 0.9162780609478365, 'recall': 0.9157814871016692, 'f1-score': 0.9160297067273812, 'support': 9226.0} | {'precision': 0.8506259119883266, 'recall': 0.9174190342085645, 'f1-score': 0.8827608193193592, 'support': 12073.0} | 0.8441   | {'precision': 0.8003061462473288, 'recall': 0.7818698222268942, 'f1-score': 0.788249681785618, 'support': 27619.0}  | {'precision': 0.8370120691110231, 'recall': 0.8440566276838408, 'f1-score': 0.8384630577202681, 'support': 27619.0} |
| No log        | 5.0   | 205  | 0.4555          | {'precision': 0.5530616263043906, 'recall': 0.6739443378119002, 'f1-score': 0.6075483940737536, 'support': 4168.0}  | {'precision': 0.7023945267958951, 'recall': 0.8587360594795539, 'f1-score': 0.7727367760819569, 'support': 2152.0} | {'precision': 0.931986531986532, 'recall': 0.9000650336006937, 'f1-score': 0.9157476841640935, 'support': 9226.0}  | {'precision': 0.9083553050277298, 'recall': 0.8275490764515862, 'f1-score': 0.8660714285714286, 'support': 12073.0} | 0.8310   | {'precision': 0.773949497528637, 'recall': 0.8150736268359334, 'f1-score': 0.7905260707228081, 'support': 27619.0}  | {'precision': 0.8465837004167056, 'recall': 0.8310221224519353, 'f1-score': 0.83637929468368, 'support': 27619.0}   |
| No log        | 6.0   | 246  | 0.4529          | {'precision': 0.5858436907520539, 'recall': 0.6672264875239923, 'f1-score': 0.6238923163208077, 'support': 4168.0}  | {'precision': 0.779319606087735, 'recall': 0.8090148698884758, 'f1-score': 0.7938896488828089, 'support': 2152.0}  | {'precision': 0.9153763440860215, 'recall': 0.9227184045089963, 'f1-score': 0.9190327107848429, 'support': 9226.0} | {'precision': 0.9016581407655672, 'recall': 0.8467655098152903, 'f1-score': 0.8733501345521336, 'support': 12073.0} | 0.8421   | {'precision': 0.7955494454228443, 'recall': 0.8114313179341887, 'f1-score': 0.8025412026351483, 'support': 27619.0} | {'precision': 0.8490485962328721, 'recall': 0.842101451899055, 'f1-score': 0.8447730063713313, 'support': 27619.0}  |
| No log        | 7.0   | 287  | 0.5058          | {'precision': 0.6189119908857875, 'recall': 0.5213531669865643, 'f1-score': 0.565959109259018, 'support': 4168.0}   | {'precision': 0.8097773475314618, 'recall': 0.7774163568773235, 'f1-score': 0.7932669511616881, 'support': 2152.0} | {'precision': 0.9237928391547137, 'recall': 0.9144808150877953, 'f1-score': 0.9191132414619532, 'support': 9226.0} | {'precision': 0.8518862808893021, 'recall': 0.9108755073304067, 'f1-score': 0.8803938835961892, 'support': 12073.0} | 0.8429   | {'precision': 0.8010921146153162, 'recall': 0.7810314615705225, 'f1-score': 0.7896832963697121, 'support': 27619.0} | {'precision': 0.8374670275215469, 'recall': 0.8428980049965603, 'f1-score': 0.8390876631549409, 'support': 27619.0} |
| No log        | 8.0   | 328  | 0.5503          | {'precision': 0.579874797124971, 'recall': 0.6000479846449136, 'f1-score': 0.5897889399834925, 'support': 4168.0}   | {'precision': 0.7640207075064711, 'recall': 0.8229553903345725, 'f1-score': 0.7923937360178971, 'support': 2152.0} | {'precision': 0.9484103877955048, 'recall': 0.882722740082376, 'f1-score': 0.9143883680458093, 'support': 9226.0}  | {'precision': 0.8650108862188534, 'recall': 0.8885115547088545, 'f1-score': 0.8766037427474054, 'support': 12073.0} | 0.8379   | {'precision': 0.7893291946614501, 'recall': 0.7985594174426791, 'f1-score': 0.793293696698651, 'support': 27619.0}  | {'precision': 0.8419711569605107, 'recall': 0.8379376516166407, 'f1-score': 0.8393807050053141, 'support': 27619.0} |
| No log        | 9.0   | 369  | 0.5586          | {'precision': 0.5553219950315307, 'recall': 0.6972168905950096, 'f1-score': 0.6182321029677693, 'support': 4168.0}  | {'precision': 0.762192490289167, 'recall': 0.820631970260223, 'f1-score': 0.7903334079212352, 'support': 2152.0}   | {'precision': 0.9405251141552512, 'recall': 0.8930197268588771, 'f1-score': 0.9161570110085622, 'support': 9226.0} | {'precision': 0.8953046246352463, 'recall': 0.8386482233082084, 'f1-score': 0.8660508083140878, 'support': 12073.0} | 0.8341   | {'precision': 0.7883360560277988, 'recall': 0.8123792027555796, 'f1-score': 0.7976933325529135, 'support': 27619.0} | {'precision': 0.8487315887907376, 'recall': 0.8340635070060466, 'f1-score': 0.8394903831187637, 'support': 27619.0} |
| No log        | 10.0  | 410  | 0.5841          | {'precision': 0.5999538319482918, 'recall': 0.6235604606525912, 'f1-score': 0.611529411764706, 'support': 4168.0}   | {'precision': 0.7489643744821872, 'recall': 0.8401486988847584, 'f1-score': 0.791940429259746, 'support': 2152.0}  | {'precision': 0.9401190748797802, 'recall': 0.8899848254931715, 'f1-score': 0.9143652561247216, 'support': 9226.0} | {'precision': 0.8760194414696433, 'recall': 0.8808084154725421, 'f1-score': 0.8784074012886173, 'support': 12073.0} | 0.8419   | {'precision': 0.7912641806949756, 'recall': 0.8086256001257658, 'f1-score': 0.7990606246094477, 'support': 27619.0} | {'precision': 0.8458706038288859, 'recall': 0.8418842101451899, 'f1-score': 0.8434069590052654, 'support': 27619.0} |
| No log        | 11.0  | 451  | 0.5806          | {'precision': 0.5889105479748754, 'recall': 0.6523512476007678, 'f1-score': 0.6190096755833807, 'support': 4168.0}  | {'precision': 0.7572354211663067, 'recall': 0.8145910780669146, 'f1-score': 0.7848668009850012, 'support': 2152.0} | {'precision': 0.9387871202639663, 'recall': 0.8943203988727509, 'f1-score': 0.91601443241743, 'support': 9226.0}   | {'precision': 0.8838460245419398, 'recall': 0.8710345398823822, 'f1-score': 0.8773935171665763, 'support': 12073.0} | 0.8414   | {'precision': 0.792194778486772, 'recall': 0.8080743161057039, 'f1-score': 0.7993211065380971, 'support': 27619.0}  | {'precision': 0.8478247878691975, 'recall': 0.8414135196784822, 'f1-score': 0.8440923556170222, 'support': 27619.0} |
| No log        | 12.0  | 492  | 0.6116          | {'precision': 0.600094540297802, 'recall': 0.6091650671785028, 'f1-score': 0.6045957852125253, 'support': 4168.0}   | {'precision': 0.7798594847775175, 'recall': 0.7736988847583643, 'f1-score': 0.7767669699090273, 'support': 2152.0} | {'precision': 0.9319225170753554, 'recall': 0.9021244309559939, 'f1-score': 0.9167814066200364, 'support': 9226.0} | {'precision': 0.8723421522480117, 'recall': 0.8903338027002402, 'f1-score': 0.8812461569993852, 'support': 12073.0} | 0.8428   | {'precision': 0.7960546735996716, 'recall': 0.7938305463982752, 'f1-score': 0.7948475796852434, 'support': 27619.0} | {'precision': 0.8439536406759814, 'recall': 0.8427531771606502, 'f1-score': 0.843226324738045, 'support': 27619.0}  |
| 0.2736        | 13.0  | 533  | 0.6151          | {'precision': 0.5982355746304244, 'recall': 0.6019673704414588, 'f1-score': 0.6000956708921311, 'support': 4168.0}  | {'precision': 0.7789473684210526, 'recall': 0.7908921933085502, 'f1-score': 0.7848743370993774, 'support': 2152.0} | {'precision': 0.9333632488220777, 'recall': 0.9017992629525254, 'f1-score': 0.9173098125689085, 'support': 9226.0} | {'precision': 0.8696251825409703, 'recall': 0.8878489190756232, 'f1-score': 0.8786425673183327, 'support': 12073.0} | 0.8418   | {'precision': 0.7950428436036313, 'recall': 0.7956269364445394, 'f1-score': 0.7952305969696873, 'support': 27619.0} | {'precision': 0.8428956433741749, 'recall': 0.8418117962272349, 'f1-score': 0.8422173277711444, 'support': 27619.0} |
| 0.2736        | 14.0  | 574  | 0.6109          | {'precision': 0.6048215551878988, 'recall': 0.6139635316698656, 'f1-score': 0.6093582569353494, 'support': 4168.0}  | {'precision': 0.7849462365591398, 'recall': 0.8141263940520446, 'f1-score': 0.7992700729927007, 'support': 2152.0} | {'precision': 0.931049822064057, 'recall': 0.9074355083459787, 'f1-score': 0.9190910088923043, 'support': 9226.0}  | {'precision': 0.8758632028937849, 'recall': 0.8824650045556199, 'f1-score': 0.8791517101951561, 'support': 12073.0} | 0.8450   | {'precision': 0.7991702041762201, 'recall': 0.8044976096558771, 'f1-score': 0.8017177622538776, 'support': 27619.0} | {'precision': 0.8463109688981529, 'recall': 0.8449618016582787, 'f1-score': 0.8455543885446015, 'support': 27619.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2