|
--- |
|
license: apache-2.0 |
|
base_model: allenai/longformer-base-4096 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- essays_su_g |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: longformer-spans |
|
results: |
|
- task: |
|
name: Token Classification |
|
type: token-classification |
|
dataset: |
|
name: essays_su_g |
|
type: essays_su_g |
|
config: spans |
|
split: train[80%:100%] |
|
args: spans |
|
metrics: |
|
- name: Accuracy |
|
type: accuracy |
|
value: 0.9388464462869763 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# longformer-spans |
|
|
|
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.2612 |
|
- B: {'precision': 0.8656716417910447, 'recall': 0.8897411313518696, 'f1-score': 0.8775413711583925, 'support': 1043.0} |
|
- I: {'precision': 0.9471240942028986, 'recall': 0.9642651296829972, 'f1-score': 0.9556177528988404, 'support': 17350.0} |
|
- O: {'precision': 0.9312169312169312, 'recall': 0.8965965748970302, 'f1-score': 0.9135788834281297, 'support': 9226.0} |
|
- Accuracy: 0.9388 |
|
- Macro avg: {'precision': 0.9146708890702916, 'recall': 0.9168676119772989, 'f1-score': 0.9155793358284542, 'support': 27619.0} |
|
- Weighted avg: {'precision': 0.9387344206602614, 'recall': 0.9388464462869763, 'f1-score': 0.9386263963728234, 'support': 27619.0} |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 13 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | B | I | O | Accuracy | Macro avg | Weighted avg | |
|
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:| |
|
| No log | 1.0 | 41 | 0.2988 | {'precision': 0.8082474226804124, 'recall': 0.37583892617449666, 'f1-score': 0.513089005235602, 'support': 1043.0} | {'precision': 0.8794226460319942, 'recall': 0.9727377521613833, 'f1-score': 0.9237295093183, 'support': 17350.0} | {'precision': 0.9257207604179781, 'recall': 0.7969867765011923, 'f1-score': 0.856543770749607, 'support': 9226.0} | 0.8915 | {'precision': 0.8711302763767949, 'recall': 0.715187818279024, 'f1-score': 0.7644540951011697, 'support': 27619.0} | {'precision': 0.8922004672916121, 'recall': 0.8914877439443861, 'f1-score': 0.885779052393972, 'support': 27619.0} | |
|
| No log | 2.0 | 82 | 0.1948 | {'precision': 0.7984293193717278, 'recall': 0.8772770853307766, 'f1-score': 0.8359981726815899, 'support': 1043.0} | {'precision': 0.9427846674182638, 'recall': 0.9639769452449568, 'f1-score': 0.9532630379025364, 'support': 17350.0} | {'precision': 0.9346158250314898, 'recall': 0.8846737481031867, 'f1-score': 0.9089592961746199, 'support': 9226.0} | 0.9342 | {'precision': 0.8919432706071605, 'recall': 0.9086425928929733, 'f1-score': 0.8994068355862487, 'support': 27619.0} | {'precision': 0.9346044882708322, 'recall': 0.9342119555378544, 'f1-score': 0.9340352028756634, 'support': 27619.0} | |
|
| No log | 3.0 | 123 | 0.1740 | {'precision': 0.848089468779124, 'recall': 0.87248322147651, 'f1-score': 0.8601134215500945, 'support': 1043.0} | {'precision': 0.9581905812670577, 'recall': 0.9510662824207493, 'f1-score': 0.9546151398571056, 'support': 17350.0} | {'precision': 0.9091689008042896, 'recall': 0.9189247778018643, 'f1-score': 0.9140208075036386, 'support': 9226.0} | 0.9374 | {'precision': 0.9051496502834904, 'recall': 0.9141580938997079, 'f1-score': 0.9095831229702794, 'support': 27619.0} | {'precision': 0.937657271434174, 'recall': 0.9373619609688982, 'f1-score': 0.9374860402341179, 'support': 27619.0} | |
|
| No log | 4.0 | 164 | 0.1780 | {'precision': 0.8725868725868726, 'recall': 0.8667305848513902, 'f1-score': 0.8696488696488696, 'support': 1043.0} | {'precision': 0.9619125269349484, 'recall': 0.9519884726224784, 'f1-score': 0.9569247704295936, 'support': 17350.0} | {'precision': 0.9102209944751382, 'recall': 0.9285714285714286, 'f1-score': 0.9193046464212898, 'support': 9226.0} | 0.9409 | {'precision': 0.9149067979989863, 'recall': 0.9157634953484323, 'f1-score': 0.9152927621665844, 'support': 27619.0} | {'precision': 0.9412719267698718, 'recall': 0.9409464499076723, 'f1-score': 0.9410620661819779, 'support': 27619.0} | |
|
| No log | 5.0 | 205 | 0.1934 | {'precision': 0.8405017921146953, 'recall': 0.8993288590604027, 'f1-score': 0.8689207966651227, 'support': 1043.0} | {'precision': 0.9406415620641562, 'recall': 0.9718155619596541, 'f1-score': 0.9559744861800141, 'support': 17350.0} | {'precision': 0.9460247143856377, 'recall': 0.8795794493821808, 'f1-score': 0.9115929004718041, 'support': 9226.0} | 0.9383 | {'precision': 0.9090560228548297, 'recall': 0.9169079568007459, 'f1-score': 0.9121627277723136, 'support': 27619.0} | {'precision': 0.9386581152797215, 'recall': 0.9382671349433361, 'f1-score': 0.93786153828516, 'support': 27619.0} | |
|
| No log | 6.0 | 246 | 0.2013 | {'precision': 0.8481481481481481, 'recall': 0.87823585810163, 'f1-score': 0.8629298162976919, 'support': 1043.0} | {'precision': 0.9306275504577037, 'recall': 0.9726801152737752, 'f1-score': 0.9511892684026604, 'support': 17350.0} | {'precision': 0.9445568114217727, 'recall': 0.8605029265120312, 'f1-score': 0.9005728546310476, 'support': 9226.0} | 0.9316 | {'precision': 0.9077775033425416, 'recall': 0.9038062999624789, 'f1-score': 0.9048973131104666, 'support': 27619.0} | {'precision': 0.9321658156029167, 'recall': 0.9316412614504508, 'f1-score': 0.9309480706039573, 'support': 27619.0} | |
|
| No log | 7.0 | 287 | 0.2083 | {'precision': 0.8447488584474886, 'recall': 0.8868648130393096, 'f1-score': 0.8652946679139383, 'support': 1043.0} | {'precision': 0.940964601271878, 'recall': 0.9636887608069165, 'f1-score': 0.9521911216150801, 'support': 17350.0} | {'precision': 0.9294117647058824, 'recall': 0.8819640147409495, 'f1-score': 0.9050664590400979, 'support': 9226.0} | 0.9335 | {'precision': 0.9050417414750829, 'recall': 0.9108391961957253, 'f1-score': 0.9075174161897054, 'support': 27619.0} | {'precision': 0.933471951649382, 'recall': 0.933487816358304, 'f1-score': 0.9331677993323372, 'support': 27619.0} | |
|
| No log | 8.0 | 328 | 0.2452 | {'precision': 0.8446251129177959, 'recall': 0.8964525407478428, 'f1-score': 0.8697674418604651, 'support': 1043.0} | {'precision': 0.9315356136376195, 'recall': 0.9716426512968299, 'f1-score': 0.9511665303128614, 'support': 17350.0} | {'precision': 0.9429590017825312, 'recall': 0.8600693691740733, 'f1-score': 0.8996088657105606, 'support': 9226.0} | 0.9315 | {'precision': 0.9063732427793155, 'recall': 0.9093881870729152, 'f1-score': 0.9068476126279624, 'support': 27619.0} | {'precision': 0.932069468113675, 'recall': 0.9315326405735183, 'f1-score': 0.9308699858008703, 'support': 27619.0} | |
|
| No log | 9.0 | 369 | 0.2303 | {'precision': 0.8538812785388128, 'recall': 0.8964525407478428, 'f1-score': 0.8746492048643593, 'support': 1043.0} | {'precision': 0.9594571080563773, 'recall': 0.9534293948126801, 'f1-score': 0.9564337544447978, 'support': 17350.0} | {'precision': 0.9145750296240439, 'recall': 0.9202254498157382, 'f1-score': 0.9173915392511751, 'support': 9226.0} | 0.9402 | {'precision': 0.909304472073078, 'recall': 0.9233691284587536, 'f1-score': 0.9161581661867775, 'support': 27619.0} | {'precision': 0.9404775053986587, 'recall': 0.9401861037691445, 'f1-score': 0.9403033817814589, 'support': 27619.0} | |
|
| No log | 10.0 | 410 | 0.2620 | {'precision': 0.8548983364140481, 'recall': 0.8868648130393096, 'f1-score': 0.8705882352941177, 'support': 1043.0} | {'precision': 0.9359622327131353, 'recall': 0.9712968299711816, 'f1-score': 0.9533022203365862, 'support': 17350.0} | {'precision': 0.9423347398030942, 'recall': 0.8714502492954693, 'f1-score': 0.9055073769568645, 'support': 9226.0} | 0.9348 | {'precision': 0.9110651029767592, 'recall': 0.9098706307686535, 'f1-score': 0.9097992775291894, 'support': 27619.0} | {'precision': 0.9350296539294, 'recall': 0.9347550599225171, 'f1-score': 0.9342129733898971, 'support': 27619.0} | |
|
| No log | 11.0 | 451 | 0.2666 | {'precision': 0.84967919340055, 'recall': 0.8887823585810163, 'f1-score': 0.8687910028116214, 'support': 1043.0} | {'precision': 0.9369943477779009, 'recall': 0.9745821325648415, 'f1-score': 0.9554186913775569, 'support': 17350.0} | {'precision': 0.9489507191700071, 'recall': 0.8724257533058747, 'f1-score': 0.9090806415179579, 'support': 9226.0} | 0.9372 | {'precision': 0.911874753449486, 'recall': 0.9119300814839107, 'f1-score': 0.9110967785690454, 'support': 27619.0} | {'precision': 0.93769096157449, 'recall': 0.9372171331329882, 'f1-score': 0.9366682830652019, 'support': 27619.0} | |
|
| No log | 12.0 | 492 | 0.2533 | {'precision': 0.859925788497217, 'recall': 0.8887823585810163, 'f1-score': 0.8741159830268741, 'support': 1043.0} | {'precision': 0.947344555914527, 'recall': 0.963342939481268, 'f1-score': 0.9552767696396423, 'support': 17350.0} | {'precision': 0.9294223420993482, 'recall': 0.8963797962280512, 'f1-score': 0.9126020745972192, 'support': 9226.0} | 0.9382 | {'precision': 0.9122308955036974, 'recall': 0.9161683647634451, 'f1-score': 0.9139982757545786, 'support': 27619.0} | {'precision': 0.9380564528305959, 'recall': 0.9381585140664036, 'f1-score': 0.9379565394756785, 'support': 27619.0} | |
|
| 0.1259 | 13.0 | 533 | 0.2612 | {'precision': 0.8656716417910447, 'recall': 0.8897411313518696, 'f1-score': 0.8775413711583925, 'support': 1043.0} | {'precision': 0.9471240942028986, 'recall': 0.9642651296829972, 'f1-score': 0.9556177528988404, 'support': 17350.0} | {'precision': 0.9312169312169312, 'recall': 0.8965965748970302, 'f1-score': 0.9135788834281297, 'support': 9226.0} | 0.9388 | {'precision': 0.9146708890702916, 'recall': 0.9168676119772989, 'f1-score': 0.9155793358284542, 'support': 27619.0} | {'precision': 0.9387344206602614, 'recall': 0.9388464462869763, 'f1-score': 0.9386263963728234, 'support': 27619.0} | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.37.2 |
|
- Pytorch 2.2.0+cu121 |
|
- Datasets 2.17.0 |
|
- Tokenizers 0.15.2 |
|
|
