Hugging Face's logo

Theoreticallyhugo
/
longformer-simple

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
df024a4
•
1 Parent(s): 8f22474

trainer: training complete at 2024-03-02 13:28:39.634997.

Browse files
Files changed (4) hide show
  1. README.md +28 -28
  2. meta_data/README_s42_e16.md +28 -28
  3. meta_data/meta_s42_e16_cvi2.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[20%:40%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.8525299930594573
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6282
36
- - Claim: {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}
37
- - Majorclaim: {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0}
38
- - O: {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}
39
- - Premise: {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0}
40
- - Accuracy: 0.8525
41
- - Macro avg: {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0}
42
- - Weighted avg: {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6017 | {'precision': 0.44954648526077096, 'recall': 0.36426274689940286, 'f1-score': 0.4024359299670135, 'support': 4354.0} | {'precision': 0.452642328312484, 'recall': 0.7547892720306514, 'f1-score': 0.5659112671560804, 'support': 2349.0} | {'precision': 0.8588395638629284, 'recall': 0.8666011787819253, 'f1-score': 0.8627029141404263, 'support': 10180.0} | {'precision': 0.8723285486443381, 'recall': 0.8179303125467324, 'f1-score': 0.8442540711584471, 'support': 13374.0} | 0.7641 | {'precision': 0.6583392315201303, 'recall': 0.7008958775646781, 'f1-score': 0.6688260456054919, 'support': 30257.0} | {'precision': 0.774369269779734, 'recall': 0.764120699342301, 'f1-score': 0.765274191732446, 'support': 30257.0} |
74
- | No log | 2.0 | 82 | 0.4534 | {'precision': 0.5700215450907972, 'recall': 0.4253559944878273, 'f1-score': 0.48717611469156913, 'support': 4354.0} | {'precision': 0.6134453781512605, 'recall': 0.8701575138356747, 'f1-score': 0.719591621193452, 'support': 2349.0} | {'precision': 0.884556428434566, 'recall': 0.9069744597249508, 'f1-score': 0.8956251818799107, 'support': 10180.0} | {'precision': 0.8789847408974165, 'recall': 0.8700463586062509, 'f1-score': 0.8744927100556139, 'support': 13374.0} | 0.8185 | {'precision': 0.7367520231435101, 'recall': 0.768133581663676, 'f1-score': 0.7442214069551364, 'support': 30257.0} | {'precision': 0.8157842273466825, 'recall': 0.8184882837029448, 'f1-score': 0.8138419333500274, 'support': 30257.0} |
75
- | No log | 3.0 | 123 | 0.4068 | {'precision': 0.5364131531168045, 'recall': 0.61070280202113, 'f1-score': 0.5711524003866395, 'support': 4354.0} | {'precision': 0.7406428885953324, 'recall': 0.7160493827160493, 'f1-score': 0.7281385281385281, 'support': 2349.0} | {'precision': 0.9368757079600453, 'recall': 0.8937131630648331, 'f1-score': 0.9147855814187321, 'support': 10180.0} | {'precision': 0.8885718576362818, 'recall': 0.8848512038283236, 'f1-score': 0.8867076277536341, 'support': 13374.0} | 0.8353 | {'precision': 0.775625901827116, 'recall': 0.7763291379075841, 'f1-score': 0.7751960344243833, 'support': 30257.0} | {'precision': 0.842663441353799, 'recall': 0.8352777869583898, 'f1-score': 0.8384354029249637, 'support': 30257.0} |
76
- | No log | 4.0 | 164 | 0.4188 | {'precision': 0.5978233358643381, 'recall': 0.5424896646761599, 'f1-score': 0.5688139674894642, 'support': 4354.0} | {'precision': 0.7550200803212851, 'recall': 0.800340570455513, 'f1-score': 0.7770200454639389, 'support': 2349.0} | {'precision': 0.9040139616055847, 'recall': 0.9159135559921414, 'f1-score': 0.9099248560554309, 'support': 10180.0} | {'precision': 0.8849800029625241, 'recall': 0.8934499775684164, 'f1-score': 0.8891948206578361, 'support': 13374.0} | 0.8433 | {'precision': 0.7854593451884331, 'recall': 0.7880484421730577, 'f1-score': 0.7862384224166675, 'support': 30257.0} | {'precision': 0.8399725571535075, 'recall': 0.8432759361470074, 'f1-score': 0.8413577905068613, 'support': 30257.0} |
77
- | No log | 5.0 | 205 | 0.4474 | {'precision': 0.5675675675675675, 'recall': 0.5884244372990354, 'f1-score': 0.5778078484438429, 'support': 4354.0} | {'precision': 0.728060263653484, 'recall': 0.8229033631332482, 'f1-score': 0.7725819344524381, 'support': 2349.0} | {'precision': 0.9372398001665279, 'recall': 0.8845776031434185, 'f1-score': 0.9101475641803114, 'support': 10180.0} | {'precision': 0.887240356083086, 'recall': 0.8942724689696426, 'f1-score': 0.8907425337007523, 'support': 13374.0} | 0.8415 | {'precision': 0.7800269968676663, 'recall': 0.7975444681363362, 'f1-score': 0.7878199701943362, 'support': 30257.0} | {'precision': 0.845703686302729, 'recall': 0.8414581749677761, 'f1-score': 0.8430665031306045, 'support': 30257.0} |
78
- | No log | 6.0 | 246 | 0.4609 | {'precision': 0.6318574213311056, 'recall': 0.5211299954065227, 'f1-score': 0.5711768407803649, 'support': 4354.0} | {'precision': 0.7856852379015861, 'recall': 0.822477650063857, 'f1-score': 0.8036605657237937, 'support': 2349.0} | {'precision': 0.9054745582697692, 'recall': 0.9212180746561887, 'f1-score': 0.9132784729999514, 'support': 10180.0} | {'precision': 0.8784115523465704, 'recall': 0.9096754897562435, 'f1-score': 0.8937702027622686, 'support': 13374.0} | 0.8509 | {'precision': 0.8003571924622578, 'recall': 0.793625302470703, 'f1-score': 0.7954715205665946, 'support': 30257.0} | {'precision': 0.8448388452449266, 'recall': 0.8508774828965198, 'f1-score': 0.8469167525043788, 'support': 30257.0} |
79
- | No log | 7.0 | 287 | 0.4865 | {'precision': 0.630808729139923, 'recall': 0.5643086816720257, 'f1-score': 0.5957085707358469, 'support': 4354.0} | {'precision': 0.7307692307692307, 'recall': 0.8573861217539378, 'f1-score': 0.7890303623898138, 'support': 2349.0} | {'precision': 0.9172843166320783, 'recall': 0.9117878192534381, 'f1-score': 0.9145278092516873, 'support': 10180.0} | {'precision': 0.8916734633350634, 'recall': 0.8992074173770002, 'f1-score': 0.8954245932764975, 'support': 13374.0} | 0.8520 | {'precision': 0.7926339349690739, 'recall': 0.8081725100141004, 'f1-score': 0.7986728339134614, 'support': 30257.0} | {'precision': 0.8502598860333094, 'recall': 0.8520011898073173, 'f1-score': 0.8504626713454606, 'support': 30257.0} |
80
- | No log | 8.0 | 328 | 0.5096 | {'precision': 0.5821842854016196, 'recall': 0.6109324758842444, 'f1-score': 0.5962120363106579, 'support': 4354.0} | {'precision': 0.774493927125506, 'recall': 0.8143891017454236, 'f1-score': 0.7939406515874662, 'support': 2349.0} | {'precision': 0.9332583810302535, 'recall': 0.8969548133595285, 'f1-score': 0.9147465437788018, 'support': 10180.0} | {'precision': 0.8931070418341521, 'recall': 0.8971138029011515, 'f1-score': 0.8951059385258131, 'support': 13374.0} | 0.8495 | {'precision': 0.7957609088478829, 'recall': 0.804847548472587, 'f1-score': 0.8000012925506848, 'support': 30257.0} | {'precision': 0.8526655157429486, 'recall': 0.8494563241563936, 'f1-score': 0.8508490740717186, 'support': 30257.0} |
81
- | No log | 9.0 | 369 | 0.5327 | {'precision': 0.6261045190608432, 'recall': 0.5695911805236564, 'f1-score': 0.5965123271196633, 'support': 4354.0} | {'precision': 0.7516019600452318, 'recall': 0.8488718603661133, 'f1-score': 0.7972810875649741, 'support': 2349.0} | {'precision': 0.9026343722860176, 'recall': 0.918860510805501, 'f1-score': 0.9106751691573773, 'support': 10180.0} | {'precision': 0.8932981927710844, 'recall': 0.8870195902497383, 'f1-score': 0.8901478202146019, 'support': 13374.0} | 0.8491 | {'precision': 0.7934097610407942, 'recall': 0.8060857854862523, 'f1-score': 0.7986541010141541, 'support': 30257.0} | {'precision': 0.846989457650438, 'recall': 0.8490927719205473, 'f1-score': 0.8475902474317125, 'support': 30257.0} |
82
- | No log | 10.0 | 410 | 0.5611 | {'precision': 0.6031589338598223, 'recall': 0.5613229214515388, 'f1-score': 0.5814894123245301, 'support': 4354.0} | {'precision': 0.7980400511291009, 'recall': 0.7973605789697744, 'f1-score': 0.7977001703577512, 'support': 2349.0} | {'precision': 0.9056966897613549, 'recall': 0.9245579567779961, 'f1-score': 0.9150301380517207, 'support': 10180.0} | {'precision': 0.8834100698054359, 'recall': 0.8894870644534171, 'f1-score': 0.8864381520119226, 'support': 13374.0} | 0.8469 | {'precision': 0.7975764361389286, 'recall': 0.7931821304131816, 'f1-score': 0.7951644681864812, 'support': 30257.0} | {'precision': 0.8439524293048358, 'recall': 0.8469114585054698, 'f1-score': 0.8452864874840641, 'support': 30257.0} |
83
- | No log | 11.0 | 451 | 0.5648 | {'precision': 0.5827433628318585, 'recall': 0.6049609554432706, 'f1-score': 0.5936443542934415, 'support': 4354.0} | {'precision': 0.7425330812854443, 'recall': 0.8361004682843763, 'f1-score': 0.7865438526231477, 'support': 2349.0} | {'precision': 0.938811369509044, 'recall': 0.8922396856581533, 'f1-score': 0.9149332661798036, 'support': 10180.0} | {'precision': 0.890064843109488, 'recall': 0.8929265739494542, 'f1-score': 0.8914934119667053, 'support': 13374.0} | 0.8468 | {'precision': 0.7885381641839586, 'recall': 0.8065569208338136, 'f1-score': 0.7966537212657745, 'support': 30257.0} | {'precision': 0.8507883056171393, 'recall': 0.8468453580989523, 'f1-score': 0.8483713709144507, 'support': 30257.0} |
84
- | No log | 12.0 | 492 | 0.6091 | {'precision': 0.6024649589173514, 'recall': 0.5725769407441433, 'f1-score': 0.5871408384361753, 'support': 4354.0} | {'precision': 0.749317738791423, 'recall': 0.8182205193699447, 'f1-score': 0.7822547822547823, 'support': 2349.0} | {'precision': 0.9307965499746321, 'recall': 0.9010805500982318, 'f1-score': 0.9156975293236835, 'support': 10180.0} | {'precision': 0.8842981239506533, 'recall': 0.9057873485868102, 'f1-score': 0.8949137517083441, 'support': 13374.0} | 0.8495 | {'precision': 0.791719342908515, 'recall': 0.7994163396997825, 'f1-score': 0.7950017254307464, 'support': 30257.0} | {'precision': 0.8489074193741941, 'recall': 0.8494563241563936, 'f1-score': 0.848871502724331, 'support': 30257.0} |
85
- | 0.2687 | 13.0 | 533 | 0.6049 | {'precision': 0.6140061306295685, 'recall': 0.5980707395498392, 'f1-score': 0.605933682373473, 'support': 4354.0} | {'precision': 0.7614920874152223, 'recall': 0.8603661132396765, 'f1-score': 0.8079152508494902, 'support': 2349.0} | {'precision': 0.921222343486457, 'recall': 0.9120825147347741, 'f1-score': 0.9166296460832224, 'support': 10180.0} | {'precision': 0.8964089437627042, 'recall': 0.8903095558546433, 'f1-score': 0.8933488389541209, 'support': 13374.0} | 0.8533 | {'precision': 0.798282376323488, 'recall': 0.8152072308447333, 'f1-score': 0.8059568545650766, 'support': 30257.0} | {'precision': 0.8536452482623537, 'recall': 0.8532570975311499, 'f1-score': 0.8531898518226914, 'support': 30257.0} |
86
- | 0.2687 | 14.0 | 574 | 0.6213 | {'precision': 0.6094716801523085, 'recall': 0.588194763435921, 'f1-score': 0.5986442262739597, 'support': 4354.0} | {'precision': 0.759737755495565, 'recall': 0.8386547467007237, 'f1-score': 0.7972480777013357, 'support': 2349.0} | {'precision': 0.9096567149664495, 'recall': 0.918860510805501, 'f1-score': 0.914235449347603, 'support': 10180.0} | {'precision': 0.8965778890659383, 'recall': 0.8835053088081352, 'f1-score': 0.88999359771024, 'support': 13374.0} | 0.8494 | {'precision': 0.7938610099200654, 'recall': 0.8073038324375702, 'f1-score': 0.8000303377582847, 'support': 30257.0} | {'precision': 0.8490399487645354, 'recall': 0.8494232739531348, 'f1-score': 0.8490241579089998, 'support': 30257.0} |
87
- | 0.2687 | 15.0 | 615 | 0.6240 | {'precision': 0.6123274631128034, 'recall': 0.5909508497932935, 'f1-score': 0.6014492753623188, 'support': 4354.0} | {'precision': 0.7543659832953683, 'recall': 0.8458918688803746, 'f1-score': 0.7975115392333936, 'support': 2349.0} | {'precision': 0.9263157894736842, 'recall': 0.9076620825147348, 'f1-score': 0.9168940709501364, 'support': 10180.0} | {'precision': 0.8923843522237096, 'recall': 0.8971885748467175, 'f1-score': 0.894780014914243, 'support': 13374.0} | 0.8527 | {'precision': 0.7963483970263914, 'recall': 0.8104233440087801, 'f1-score': 0.802658725115023, 'support': 30257.0} | {'precision': 0.8527852243327482, 'recall': 0.8526621938724923, 'f1-score': 0.8524584166415128, 'support': 30257.0} |
88
- | 0.2687 | 16.0 | 656 | 0.6282 | {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0} | {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0} | {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0} | {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0} | 0.8525 | {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0} | {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[40%:60%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.844576254146979
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6960
36
+ - Claim: {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0}
37
+ - Majorclaim: {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0}
38
+ - O: {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0}
39
+ - Premise: {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0}
40
+ - Accuracy: 0.8446
41
+ - Macro avg: {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0}
42
+ - Weighted avg: {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.5869 | {'precision': 0.535017852238396, 'recall': 0.4274742154926487, 'f1-score': 0.47523786289338865, 'support': 4557.0} | {'precision': 0.5474121647147714, 'recall': 0.6386073159982371, 'f1-score': 0.5895036615134255, 'support': 2269.0} | {'precision': 0.845403060609712, 'recall': 0.8272609362103526, 'f1-score': 0.8362336114421931, 'support': 8481.0} | {'precision': 0.850072112232857, 'recall': 0.8921838447777625, 'f1-score': 0.8706190412246543, 'support': 14534.0} | 0.7835 | {'precision': 0.694476297448934, 'recall': 0.6963815781197502, 'f1-score': 0.6928985442684154, 'support': 29841.0} | {'precision': 0.7776202536983177, 'recall': 0.7834858081163499, 'f1-score': 0.7790930985214805, 'support': 29841.0} |
74
+ | No log | 2.0 | 82 | 0.4861 | {'precision': 0.6276500447894894, 'recall': 0.4612683783190696, 'f1-score': 0.5317480394636985, 'support': 4557.0} | {'precision': 0.6855268552685527, 'recall': 0.7368884971353019, 'f1-score': 0.7102803738317758, 'support': 2269.0} | {'precision': 0.872397977184523, 'recall': 0.8746610069567268, 'f1-score': 0.8735280263777673, 'support': 8481.0} | {'precision': 0.8556913183279743, 'recall': 0.9155084629145452, 'f1-score': 0.884589815184151, 'support': 14534.0} | 0.8210 | {'precision': 0.7603165488926349, 'recall': 0.7470815863314109, 'f1-score': 0.7500365637143481, 'support': 29841.0} | {'precision': 0.8126767385071133, 'recall': 0.820951040514728, 'f1-score': 0.8143098940939201, 'support': 29841.0} |
75
+ | No log | 3.0 | 123 | 0.4651 | {'precision': 0.5519389190275267, 'recall': 0.6028088654816766, 'f1-score': 0.5762534088525278, 'support': 4557.0} | {'precision': 0.6811542572141076, 'recall': 0.8426619656236227, 'f1-score': 0.7533490937746257, 'support': 2269.0} | {'precision': 0.9045047256658892, 'recall': 0.868883386393114, 'f1-score': 0.8863363002165023, 'support': 8481.0} | {'precision': 0.8910855499640546, 'recall': 0.8528278519333975, 'f1-score': 0.8715370552664885, 'support': 14534.0} | 0.8184 | {'precision': 0.7571708629678946, 'recall': 0.7917955173579527, 'f1-score': 0.7718689645275361, 'support': 29841.0} | {'precision': 0.8271460951435015, 'recall': 0.8184377199155525, 'f1-score': 0.8216639389194362, 'support': 29841.0} |
76
+ | No log | 4.0 | 164 | 0.4685 | {'precision': 0.5727291118753793, 'recall': 0.6212420452051789, 'f1-score': 0.5960000000000001, 'support': 4557.0} | {'precision': 0.7450166112956811, 'recall': 0.7906566769501984, 'f1-score': 0.7671584348941629, 'support': 2269.0} | {'precision': 0.8872651356993737, 'recall': 0.9020162716660771, 'f1-score': 0.8945798982634625, 'support': 8481.0} | {'precision': 0.8981107585809057, 'recall': 0.8569561029310582, 'f1-score': 0.8770509119076122, 'support': 14534.0} | 0.8287 | {'precision': 0.7757804043628349, 'recall': 0.792717774188128, 'f1-score': 0.7836973112663095, 'support': 29841.0} | {'precision': 0.8336988249364055, 'recall': 0.8287255789015113, 'f1-score': 0.8307578351802057, 'support': 29841.0} |
77
+ | No log | 5.0 | 205 | 0.4714 | {'precision': 0.6111239326102008, 'recall': 0.5810840465218345, 'f1-score': 0.5957255343082115, 'support': 4557.0} | {'precision': 0.7859340659340659, 'recall': 0.7880123402379903, 'f1-score': 0.7869718309859155, 'support': 2269.0} | {'precision': 0.8915166490175315, 'recall': 0.8934087961325315, 'f1-score': 0.8924617196702003, 'support': 8481.0} | {'precision': 0.8798018189222208, 'recall': 0.8919086280445852, 'f1-score': 0.8858138581385815, 'support': 14534.0} | 0.8370 | {'precision': 0.7920941166210047, 'recall': 0.7886034527342354, 'f1-score': 0.7902432357757272, 'support': 29841.0} | {'precision': 0.8349642603479214, 'recall': 0.8369692704668074, 'f1-score': 0.8358884354766487, 'support': 29841.0} |
78
+ | No log | 6.0 | 246 | 0.5037 | {'precision': 0.5850368809272919, 'recall': 0.6091727013385999, 'f1-score': 0.5968608901311546, 'support': 4557.0} | {'precision': 0.8594594594594595, 'recall': 0.7708241516086382, 'f1-score': 0.8127323420074349, 'support': 2269.0} | {'precision': 0.8999051233396584, 'recall': 0.8947058129937507, 'f1-score': 0.8972979364985514, 'support': 8481.0} | {'precision': 0.8796910246770114, 'recall': 0.8854410348149168, 'f1-score': 0.8825566642663649, 'support': 14534.0} | 0.8372 | {'precision': 0.8060231221008552, 'recall': 0.7900359251889764, 'f1-score': 0.7973619582258764, 'support': 29841.0} | {'precision': 0.8389012192486348, 'recall': 0.8371703361147415, 'f1-score': 0.8378086229762441, 'support': 29841.0} |
79
+ | No log | 7.0 | 287 | 0.5330 | {'precision': 0.6130337078651685, 'recall': 0.5986394557823129, 'f1-score': 0.6057510824913955, 'support': 4557.0} | {'precision': 0.8630921395106715, 'recall': 0.7307183781401498, 'f1-score': 0.7914081145584726, 'support': 2269.0} | {'precision': 0.8824737562756733, 'recall': 0.9119207640608419, 'f1-score': 0.8969556393157437, 'support': 8481.0} | {'precision': 0.880592955256358, 'recall': 0.8910141736617586, 'f1-score': 0.8857729138166894, 'support': 14534.0} | 0.8401 | {'precision': 0.8097981397269679, 'recall': 0.7830731929112658, 'f1-score': 0.7949719375455753, 'support': 29841.0} | {'precision': 0.8389379916879856, 'recall': 0.8401192989511075, 'f1-score': 0.8390140076168712, 'support': 29841.0} |
80
+ | No log | 8.0 | 328 | 0.5759 | {'precision': 0.599912453490917, 'recall': 0.6014922097871407, 'f1-score': 0.6007012930089853, 'support': 4557.0} | {'precision': 0.8645575877409788, 'recall': 0.7708241516086382, 'f1-score': 0.8150046598322461, 'support': 2269.0} | {'precision': 0.9148230088495575, 'recall': 0.8776087725504068, 'f1-score': 0.8958295721249322, 'support': 8481.0} | {'precision': 0.8694501422616291, 'recall': 0.9040869684876841, 'f1-score': 0.8864303302189092, 'support': 14534.0} | 0.8402 | {'precision': 0.8121857980857706, 'recall': 0.7885030256084674, 'f1-score': 0.7994914637962682, 'support': 29841.0} | {'precision': 0.8408124567818104, 'recall': 0.8402198317750745, 'f1-score': 0.8400372100799064, 'support': 29841.0} |
81
+ | No log | 9.0 | 369 | 0.5976 | {'precision': 0.6026747195858498, 'recall': 0.6131226684222076, 'f1-score': 0.6078538018057218, 'support': 4557.0} | {'precision': 0.8060156931124673, 'recall': 0.8148964301454386, 'f1-score': 0.8104317335086566, 'support': 2269.0} | {'precision': 0.9120731707317074, 'recall': 0.8818535550053059, 'f1-score': 0.8967088304058509, 'support': 8481.0} | {'precision': 0.8804975868397797, 'recall': 0.8912205862116417, 'f1-score': 0.8858266370319713, 'support': 14534.0} | 0.8403 | {'precision': 0.800315292567451, 'recall': 0.8002733099461484, 'f1-score': 0.8002052506880502, 'support': 29841.0} | {'precision': 0.8413820848138425, 'recall': 0.8402868536577193, 'f1-score': 0.8407376197665799, 'support': 29841.0} |
82
+ | No log | 10.0 | 410 | 0.6327 | {'precision': 0.6153846153846154, 'recall': 0.6179504059688391, 'f1-score': 0.6166648417825469, 'support': 4557.0} | {'precision': 0.7641955835962145, 'recall': 0.8541207580431909, 'f1-score': 0.8066597294484912, 'support': 2269.0} | {'precision': 0.9150908869098451, 'recall': 0.8844475887277443, 'f1-score': 0.8995083343326538, 'support': 8481.0} | {'precision': 0.8894852738783374, 'recall': 0.8893628732626944, 'f1-score': 0.8894240693593889, 'support': 14534.0} | 0.8438 | {'precision': 0.7960390899422531, 'recall': 0.8114704065006171, 'f1-score': 0.8030642437307701, 'support': 29841.0} | {'precision': 0.8453782465037248, 'recall': 0.8438390134378875, 'f1-score': 0.8443440976397001, 'support': 29841.0} |
83
+ | No log | 11.0 | 451 | 0.6347 | {'precision': 0.5944913550462404, 'recall': 0.6488918147904323, 'f1-score': 0.6205015213513796, 'support': 4557.0} | {'precision': 0.78500823723229, 'recall': 0.8400176289114147, 'f1-score': 0.8115818607621886, 'support': 2269.0} | {'precision': 0.9032919329555047, 'recall': 0.8832684824902723, 'f1-score': 0.8931679980922858, 'support': 8481.0} | {'precision': 0.8962250812950657, 'recall': 0.872299435805697, 'f1-score': 0.8841004184100418, 'support': 14534.0} | 0.8388 | {'precision': 0.7947541516322751, 'recall': 0.8111193404994541, 'f1-score': 0.802337949653974, 'support': 29841.0} | {'precision': 0.843699440707882, 'recall': 0.8388458831808585, 'f1-score': 0.8409094181783406, 'support': 29841.0} |
84
+ | No log | 12.0 | 492 | 0.6513 | {'precision': 0.6110076557003932, 'recall': 0.6480140443274084, 'f1-score': 0.6289669861554845, 'support': 4557.0} | {'precision': 0.803946803946804, 'recall': 0.8259144997796386, 'f1-score': 0.8147826086956521, 'support': 2269.0} | {'precision': 0.901905099988167, 'recall': 0.8987147742011555, 'f1-score': 0.9003071107961257, 'support': 8481.0} | {'precision': 0.8980036552790664, 'recall': 0.8789734415852484, 'f1-score': 0.8883866481223922, 'support': 14534.0} | 0.8453 | {'precision': 0.8037158037286076, 'recall': 0.8129041899733627, 'f1-score': 0.8081108384424137, 'support': 29841.0} | {'precision': 0.8481337577161484, 'recall': 0.8452799839147481, 'f1-score': 0.8465621274593268, 'support': 29841.0} |
85
+ | 0.2641 | 13.0 | 533 | 0.6643 | {'precision': 0.6193424423569599, 'recall': 0.6366030283080975, 'f1-score': 0.6278541283410888, 'support': 4557.0} | {'precision': 0.8395522388059702, 'recall': 0.7933010136624064, 'f1-score': 0.8157715839564923, 'support': 2269.0} | {'precision': 0.8938955172014363, 'recall': 0.9099162834571395, 'f1-score': 0.9018347551712048, 'support': 8481.0} | {'precision': 0.8938108484005564, 'recall': 0.8843401678822073, 'f1-score': 0.8890502870581725, 'support': 14534.0} | 0.8469 | {'precision': 0.8116502616912307, 'recall': 0.8060401233274627, 'f1-score': 0.8086276886317396, 'support': 29841.0} | {'precision': 0.8477953919677784, 'recall': 0.8468549981568982, 'f1-score': 0.8472247718762136, 'support': 29841.0} |
86
+ | 0.2641 | 14.0 | 574 | 0.6926 | {'precision': 0.5876997774630791, 'recall': 0.6374807987711214, 'f1-score': 0.6115789473684211, 'support': 4557.0} | {'precision': 0.8265213442325159, 'recall': 0.8021154693697664, 'f1-score': 0.814135540147618, 'support': 2269.0} | {'precision': 0.8866728153101222, 'recall': 0.9068506072397123, 'f1-score': 0.8966482075196736, 'support': 8481.0} | {'precision': 0.8985879332477535, 'recall': 0.8669327095087381, 'f1-score': 0.8824765373301583, 'support': 14534.0} | 0.8383 | {'precision': 0.7998704675633677, 'recall': 0.8033448962223346, 'f1-score': 0.8012098080914677, 'support': 29841.0} | {'precision': 0.8422463719188642, 'recall': 0.8383097081197011, 'f1-score': 0.8399392193721293, 'support': 29841.0} |
87
+ | 0.2641 | 15.0 | 615 | 0.6816 | {'precision': 0.594445578925873, 'recall': 0.6387974544656573, 'f1-score': 0.6158239898455681, 'support': 4557.0} | {'precision': 0.8150388936905791, 'recall': 0.8312031732040547, 'f1-score': 0.8230416757582371, 'support': 2269.0} | {'precision': 0.9040445973194164, 'recall': 0.8987147742011555, 'f1-score': 0.901371807000946, 'support': 8481.0} | {'precision': 0.8959785900415522, 'recall': 0.8753268198706481, 'f1-score': 0.885532314760032, 'support': 14534.0} | 0.8425 | {'precision': 0.8023769149943552, 'recall': 0.8110105554353789, 'f1-score': 0.8064424468411957, 'support': 29841.0} | {'precision': 0.8460697299178652, 'recall': 0.8424985757849938, 'f1-score': 0.8440954539700084, 'support': 29841.0} |
88
+ | 0.2641 | 16.0 | 656 | 0.6960 | {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0} | {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0} | {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0} | {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0} | 0.8446 | {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0} | {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0} |
89
 
90
 
91
  ### Framework versions
meta_data/README_s42_e16.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[20%:40%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.8525299930594573
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6282
36
- - Claim: {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}
37
- - Majorclaim: {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0}
38
- - O: {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}
39
- - Premise: {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0}
40
- - Accuracy: 0.8525
41
- - Macro avg: {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0}
42
- - Weighted avg: {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6017 | {'precision': 0.44954648526077096, 'recall': 0.36426274689940286, 'f1-score': 0.4024359299670135, 'support': 4354.0} | {'precision': 0.452642328312484, 'recall': 0.7547892720306514, 'f1-score': 0.5659112671560804, 'support': 2349.0} | {'precision': 0.8588395638629284, 'recall': 0.8666011787819253, 'f1-score': 0.8627029141404263, 'support': 10180.0} | {'precision': 0.8723285486443381, 'recall': 0.8179303125467324, 'f1-score': 0.8442540711584471, 'support': 13374.0} | 0.7641 | {'precision': 0.6583392315201303, 'recall': 0.7008958775646781, 'f1-score': 0.6688260456054919, 'support': 30257.0} | {'precision': 0.774369269779734, 'recall': 0.764120699342301, 'f1-score': 0.765274191732446, 'support': 30257.0} |
74
- | No log | 2.0 | 82 | 0.4534 | {'precision': 0.5700215450907972, 'recall': 0.4253559944878273, 'f1-score': 0.48717611469156913, 'support': 4354.0} | {'precision': 0.6134453781512605, 'recall': 0.8701575138356747, 'f1-score': 0.719591621193452, 'support': 2349.0} | {'precision': 0.884556428434566, 'recall': 0.9069744597249508, 'f1-score': 0.8956251818799107, 'support': 10180.0} | {'precision': 0.8789847408974165, 'recall': 0.8700463586062509, 'f1-score': 0.8744927100556139, 'support': 13374.0} | 0.8185 | {'precision': 0.7367520231435101, 'recall': 0.768133581663676, 'f1-score': 0.7442214069551364, 'support': 30257.0} | {'precision': 0.8157842273466825, 'recall': 0.8184882837029448, 'f1-score': 0.8138419333500274, 'support': 30257.0} |
75
- | No log | 3.0 | 123 | 0.4068 | {'precision': 0.5364131531168045, 'recall': 0.61070280202113, 'f1-score': 0.5711524003866395, 'support': 4354.0} | {'precision': 0.7406428885953324, 'recall': 0.7160493827160493, 'f1-score': 0.7281385281385281, 'support': 2349.0} | {'precision': 0.9368757079600453, 'recall': 0.8937131630648331, 'f1-score': 0.9147855814187321, 'support': 10180.0} | {'precision': 0.8885718576362818, 'recall': 0.8848512038283236, 'f1-score': 0.8867076277536341, 'support': 13374.0} | 0.8353 | {'precision': 0.775625901827116, 'recall': 0.7763291379075841, 'f1-score': 0.7751960344243833, 'support': 30257.0} | {'precision': 0.842663441353799, 'recall': 0.8352777869583898, 'f1-score': 0.8384354029249637, 'support': 30257.0} |
76
- | No log | 4.0 | 164 | 0.4188 | {'precision': 0.5978233358643381, 'recall': 0.5424896646761599, 'f1-score': 0.5688139674894642, 'support': 4354.0} | {'precision': 0.7550200803212851, 'recall': 0.800340570455513, 'f1-score': 0.7770200454639389, 'support': 2349.0} | {'precision': 0.9040139616055847, 'recall': 0.9159135559921414, 'f1-score': 0.9099248560554309, 'support': 10180.0} | {'precision': 0.8849800029625241, 'recall': 0.8934499775684164, 'f1-score': 0.8891948206578361, 'support': 13374.0} | 0.8433 | {'precision': 0.7854593451884331, 'recall': 0.7880484421730577, 'f1-score': 0.7862384224166675, 'support': 30257.0} | {'precision': 0.8399725571535075, 'recall': 0.8432759361470074, 'f1-score': 0.8413577905068613, 'support': 30257.0} |
77
- | No log | 5.0 | 205 | 0.4474 | {'precision': 0.5675675675675675, 'recall': 0.5884244372990354, 'f1-score': 0.5778078484438429, 'support': 4354.0} | {'precision': 0.728060263653484, 'recall': 0.8229033631332482, 'f1-score': 0.7725819344524381, 'support': 2349.0} | {'precision': 0.9372398001665279, 'recall': 0.8845776031434185, 'f1-score': 0.9101475641803114, 'support': 10180.0} | {'precision': 0.887240356083086, 'recall': 0.8942724689696426, 'f1-score': 0.8907425337007523, 'support': 13374.0} | 0.8415 | {'precision': 0.7800269968676663, 'recall': 0.7975444681363362, 'f1-score': 0.7878199701943362, 'support': 30257.0} | {'precision': 0.845703686302729, 'recall': 0.8414581749677761, 'f1-score': 0.8430665031306045, 'support': 30257.0} |
78
- | No log | 6.0 | 246 | 0.4609 | {'precision': 0.6318574213311056, 'recall': 0.5211299954065227, 'f1-score': 0.5711768407803649, 'support': 4354.0} | {'precision': 0.7856852379015861, 'recall': 0.822477650063857, 'f1-score': 0.8036605657237937, 'support': 2349.0} | {'precision': 0.9054745582697692, 'recall': 0.9212180746561887, 'f1-score': 0.9132784729999514, 'support': 10180.0} | {'precision': 0.8784115523465704, 'recall': 0.9096754897562435, 'f1-score': 0.8937702027622686, 'support': 13374.0} | 0.8509 | {'precision': 0.8003571924622578, 'recall': 0.793625302470703, 'f1-score': 0.7954715205665946, 'support': 30257.0} | {'precision': 0.8448388452449266, 'recall': 0.8508774828965198, 'f1-score': 0.8469167525043788, 'support': 30257.0} |
79
- | No log | 7.0 | 287 | 0.4865 | {'precision': 0.630808729139923, 'recall': 0.5643086816720257, 'f1-score': 0.5957085707358469, 'support': 4354.0} | {'precision': 0.7307692307692307, 'recall': 0.8573861217539378, 'f1-score': 0.7890303623898138, 'support': 2349.0} | {'precision': 0.9172843166320783, 'recall': 0.9117878192534381, 'f1-score': 0.9145278092516873, 'support': 10180.0} | {'precision': 0.8916734633350634, 'recall': 0.8992074173770002, 'f1-score': 0.8954245932764975, 'support': 13374.0} | 0.8520 | {'precision': 0.7926339349690739, 'recall': 0.8081725100141004, 'f1-score': 0.7986728339134614, 'support': 30257.0} | {'precision': 0.8502598860333094, 'recall': 0.8520011898073173, 'f1-score': 0.8504626713454606, 'support': 30257.0} |
80
- | No log | 8.0 | 328 | 0.5096 | {'precision': 0.5821842854016196, 'recall': 0.6109324758842444, 'f1-score': 0.5962120363106579, 'support': 4354.0} | {'precision': 0.774493927125506, 'recall': 0.8143891017454236, 'f1-score': 0.7939406515874662, 'support': 2349.0} | {'precision': 0.9332583810302535, 'recall': 0.8969548133595285, 'f1-score': 0.9147465437788018, 'support': 10180.0} | {'precision': 0.8931070418341521, 'recall': 0.8971138029011515, 'f1-score': 0.8951059385258131, 'support': 13374.0} | 0.8495 | {'precision': 0.7957609088478829, 'recall': 0.804847548472587, 'f1-score': 0.8000012925506848, 'support': 30257.0} | {'precision': 0.8526655157429486, 'recall': 0.8494563241563936, 'f1-score': 0.8508490740717186, 'support': 30257.0} |
81
- | No log | 9.0 | 369 | 0.5327 | {'precision': 0.6261045190608432, 'recall': 0.5695911805236564, 'f1-score': 0.5965123271196633, 'support': 4354.0} | {'precision': 0.7516019600452318, 'recall': 0.8488718603661133, 'f1-score': 0.7972810875649741, 'support': 2349.0} | {'precision': 0.9026343722860176, 'recall': 0.918860510805501, 'f1-score': 0.9106751691573773, 'support': 10180.0} | {'precision': 0.8932981927710844, 'recall': 0.8870195902497383, 'f1-score': 0.8901478202146019, 'support': 13374.0} | 0.8491 | {'precision': 0.7934097610407942, 'recall': 0.8060857854862523, 'f1-score': 0.7986541010141541, 'support': 30257.0} | {'precision': 0.846989457650438, 'recall': 0.8490927719205473, 'f1-score': 0.8475902474317125, 'support': 30257.0} |
82
- | No log | 10.0 | 410 | 0.5611 | {'precision': 0.6031589338598223, 'recall': 0.5613229214515388, 'f1-score': 0.5814894123245301, 'support': 4354.0} | {'precision': 0.7980400511291009, 'recall': 0.7973605789697744, 'f1-score': 0.7977001703577512, 'support': 2349.0} | {'precision': 0.9056966897613549, 'recall': 0.9245579567779961, 'f1-score': 0.9150301380517207, 'support': 10180.0} | {'precision': 0.8834100698054359, 'recall': 0.8894870644534171, 'f1-score': 0.8864381520119226, 'support': 13374.0} | 0.8469 | {'precision': 0.7975764361389286, 'recall': 0.7931821304131816, 'f1-score': 0.7951644681864812, 'support': 30257.0} | {'precision': 0.8439524293048358, 'recall': 0.8469114585054698, 'f1-score': 0.8452864874840641, 'support': 30257.0} |
83
- | No log | 11.0 | 451 | 0.5648 | {'precision': 0.5827433628318585, 'recall': 0.6049609554432706, 'f1-score': 0.5936443542934415, 'support': 4354.0} | {'precision': 0.7425330812854443, 'recall': 0.8361004682843763, 'f1-score': 0.7865438526231477, 'support': 2349.0} | {'precision': 0.938811369509044, 'recall': 0.8922396856581533, 'f1-score': 0.9149332661798036, 'support': 10180.0} | {'precision': 0.890064843109488, 'recall': 0.8929265739494542, 'f1-score': 0.8914934119667053, 'support': 13374.0} | 0.8468 | {'precision': 0.7885381641839586, 'recall': 0.8065569208338136, 'f1-score': 0.7966537212657745, 'support': 30257.0} | {'precision': 0.8507883056171393, 'recall': 0.8468453580989523, 'f1-score': 0.8483713709144507, 'support': 30257.0} |
84
- | No log | 12.0 | 492 | 0.6091 | {'precision': 0.6024649589173514, 'recall': 0.5725769407441433, 'f1-score': 0.5871408384361753, 'support': 4354.0} | {'precision': 0.749317738791423, 'recall': 0.8182205193699447, 'f1-score': 0.7822547822547823, 'support': 2349.0} | {'precision': 0.9307965499746321, 'recall': 0.9010805500982318, 'f1-score': 0.9156975293236835, 'support': 10180.0} | {'precision': 0.8842981239506533, 'recall': 0.9057873485868102, 'f1-score': 0.8949137517083441, 'support': 13374.0} | 0.8495 | {'precision': 0.791719342908515, 'recall': 0.7994163396997825, 'f1-score': 0.7950017254307464, 'support': 30257.0} | {'precision': 0.8489074193741941, 'recall': 0.8494563241563936, 'f1-score': 0.848871502724331, 'support': 30257.0} |
85
- | 0.2687 | 13.0 | 533 | 0.6049 | {'precision': 0.6140061306295685, 'recall': 0.5980707395498392, 'f1-score': 0.605933682373473, 'support': 4354.0} | {'precision': 0.7614920874152223, 'recall': 0.8603661132396765, 'f1-score': 0.8079152508494902, 'support': 2349.0} | {'precision': 0.921222343486457, 'recall': 0.9120825147347741, 'f1-score': 0.9166296460832224, 'support': 10180.0} | {'precision': 0.8964089437627042, 'recall': 0.8903095558546433, 'f1-score': 0.8933488389541209, 'support': 13374.0} | 0.8533 | {'precision': 0.798282376323488, 'recall': 0.8152072308447333, 'f1-score': 0.8059568545650766, 'support': 30257.0} | {'precision': 0.8536452482623537, 'recall': 0.8532570975311499, 'f1-score': 0.8531898518226914, 'support': 30257.0} |
86
- | 0.2687 | 14.0 | 574 | 0.6213 | {'precision': 0.6094716801523085, 'recall': 0.588194763435921, 'f1-score': 0.5986442262739597, 'support': 4354.0} | {'precision': 0.759737755495565, 'recall': 0.8386547467007237, 'f1-score': 0.7972480777013357, 'support': 2349.0} | {'precision': 0.9096567149664495, 'recall': 0.918860510805501, 'f1-score': 0.914235449347603, 'support': 10180.0} | {'precision': 0.8965778890659383, 'recall': 0.8835053088081352, 'f1-score': 0.88999359771024, 'support': 13374.0} | 0.8494 | {'precision': 0.7938610099200654, 'recall': 0.8073038324375702, 'f1-score': 0.8000303377582847, 'support': 30257.0} | {'precision': 0.8490399487645354, 'recall': 0.8494232739531348, 'f1-score': 0.8490241579089998, 'support': 30257.0} |
87
- | 0.2687 | 15.0 | 615 | 0.6240 | {'precision': 0.6123274631128034, 'recall': 0.5909508497932935, 'f1-score': 0.6014492753623188, 'support': 4354.0} | {'precision': 0.7543659832953683, 'recall': 0.8458918688803746, 'f1-score': 0.7975115392333936, 'support': 2349.0} | {'precision': 0.9263157894736842, 'recall': 0.9076620825147348, 'f1-score': 0.9168940709501364, 'support': 10180.0} | {'precision': 0.8923843522237096, 'recall': 0.8971885748467175, 'f1-score': 0.894780014914243, 'support': 13374.0} | 0.8527 | {'precision': 0.7963483970263914, 'recall': 0.8104233440087801, 'f1-score': 0.802658725115023, 'support': 30257.0} | {'precision': 0.8527852243327482, 'recall': 0.8526621938724923, 'f1-score': 0.8524584166415128, 'support': 30257.0} |
88
- | 0.2687 | 16.0 | 656 | 0.6282 | {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0} | {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0} | {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0} | {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0} | 0.8525 | {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0} | {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[40%:60%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.844576254146979
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6960
36
+ - Claim: {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0}
37
+ - Majorclaim: {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0}
38
+ - O: {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0}
39
+ - Premise: {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0}
40
+ - Accuracy: 0.8446
41
+ - Macro avg: {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0}
42
+ - Weighted avg: {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.5869 | {'precision': 0.535017852238396, 'recall': 0.4274742154926487, 'f1-score': 0.47523786289338865, 'support': 4557.0} | {'precision': 0.5474121647147714, 'recall': 0.6386073159982371, 'f1-score': 0.5895036615134255, 'support': 2269.0} | {'precision': 0.845403060609712, 'recall': 0.8272609362103526, 'f1-score': 0.8362336114421931, 'support': 8481.0} | {'precision': 0.850072112232857, 'recall': 0.8921838447777625, 'f1-score': 0.8706190412246543, 'support': 14534.0} | 0.7835 | {'precision': 0.694476297448934, 'recall': 0.6963815781197502, 'f1-score': 0.6928985442684154, 'support': 29841.0} | {'precision': 0.7776202536983177, 'recall': 0.7834858081163499, 'f1-score': 0.7790930985214805, 'support': 29841.0} |
74
+ | No log | 2.0 | 82 | 0.4861 | {'precision': 0.6276500447894894, 'recall': 0.4612683783190696, 'f1-score': 0.5317480394636985, 'support': 4557.0} | {'precision': 0.6855268552685527, 'recall': 0.7368884971353019, 'f1-score': 0.7102803738317758, 'support': 2269.0} | {'precision': 0.872397977184523, 'recall': 0.8746610069567268, 'f1-score': 0.8735280263777673, 'support': 8481.0} | {'precision': 0.8556913183279743, 'recall': 0.9155084629145452, 'f1-score': 0.884589815184151, 'support': 14534.0} | 0.8210 | {'precision': 0.7603165488926349, 'recall': 0.7470815863314109, 'f1-score': 0.7500365637143481, 'support': 29841.0} | {'precision': 0.8126767385071133, 'recall': 0.820951040514728, 'f1-score': 0.8143098940939201, 'support': 29841.0} |
75
+ | No log | 3.0 | 123 | 0.4651 | {'precision': 0.5519389190275267, 'recall': 0.6028088654816766, 'f1-score': 0.5762534088525278, 'support': 4557.0} | {'precision': 0.6811542572141076, 'recall': 0.8426619656236227, 'f1-score': 0.7533490937746257, 'support': 2269.0} | {'precision': 0.9045047256658892, 'recall': 0.868883386393114, 'f1-score': 0.8863363002165023, 'support': 8481.0} | {'precision': 0.8910855499640546, 'recall': 0.8528278519333975, 'f1-score': 0.8715370552664885, 'support': 14534.0} | 0.8184 | {'precision': 0.7571708629678946, 'recall': 0.7917955173579527, 'f1-score': 0.7718689645275361, 'support': 29841.0} | {'precision': 0.8271460951435015, 'recall': 0.8184377199155525, 'f1-score': 0.8216639389194362, 'support': 29841.0} |
76
+ | No log | 4.0 | 164 | 0.4685 | {'precision': 0.5727291118753793, 'recall': 0.6212420452051789, 'f1-score': 0.5960000000000001, 'support': 4557.0} | {'precision': 0.7450166112956811, 'recall': 0.7906566769501984, 'f1-score': 0.7671584348941629, 'support': 2269.0} | {'precision': 0.8872651356993737, 'recall': 0.9020162716660771, 'f1-score': 0.8945798982634625, 'support': 8481.0} | {'precision': 0.8981107585809057, 'recall': 0.8569561029310582, 'f1-score': 0.8770509119076122, 'support': 14534.0} | 0.8287 | {'precision': 0.7757804043628349, 'recall': 0.792717774188128, 'f1-score': 0.7836973112663095, 'support': 29841.0} | {'precision': 0.8336988249364055, 'recall': 0.8287255789015113, 'f1-score': 0.8307578351802057, 'support': 29841.0} |
77
+ | No log | 5.0 | 205 | 0.4714 | {'precision': 0.6111239326102008, 'recall': 0.5810840465218345, 'f1-score': 0.5957255343082115, 'support': 4557.0} | {'precision': 0.7859340659340659, 'recall': 0.7880123402379903, 'f1-score': 0.7869718309859155, 'support': 2269.0} | {'precision': 0.8915166490175315, 'recall': 0.8934087961325315, 'f1-score': 0.8924617196702003, 'support': 8481.0} | {'precision': 0.8798018189222208, 'recall': 0.8919086280445852, 'f1-score': 0.8858138581385815, 'support': 14534.0} | 0.8370 | {'precision': 0.7920941166210047, 'recall': 0.7886034527342354, 'f1-score': 0.7902432357757272, 'support': 29841.0} | {'precision': 0.8349642603479214, 'recall': 0.8369692704668074, 'f1-score': 0.8358884354766487, 'support': 29841.0} |
78
+ | No log | 6.0 | 246 | 0.5037 | {'precision': 0.5850368809272919, 'recall': 0.6091727013385999, 'f1-score': 0.5968608901311546, 'support': 4557.0} | {'precision': 0.8594594594594595, 'recall': 0.7708241516086382, 'f1-score': 0.8127323420074349, 'support': 2269.0} | {'precision': 0.8999051233396584, 'recall': 0.8947058129937507, 'f1-score': 0.8972979364985514, 'support': 8481.0} | {'precision': 0.8796910246770114, 'recall': 0.8854410348149168, 'f1-score': 0.8825566642663649, 'support': 14534.0} | 0.8372 | {'precision': 0.8060231221008552, 'recall': 0.7900359251889764, 'f1-score': 0.7973619582258764, 'support': 29841.0} | {'precision': 0.8389012192486348, 'recall': 0.8371703361147415, 'f1-score': 0.8378086229762441, 'support': 29841.0} |
79
+ | No log | 7.0 | 287 | 0.5330 | {'precision': 0.6130337078651685, 'recall': 0.5986394557823129, 'f1-score': 0.6057510824913955, 'support': 4557.0} | {'precision': 0.8630921395106715, 'recall': 0.7307183781401498, 'f1-score': 0.7914081145584726, 'support': 2269.0} | {'precision': 0.8824737562756733, 'recall': 0.9119207640608419, 'f1-score': 0.8969556393157437, 'support': 8481.0} | {'precision': 0.880592955256358, 'recall': 0.8910141736617586, 'f1-score': 0.8857729138166894, 'support': 14534.0} | 0.8401 | {'precision': 0.8097981397269679, 'recall': 0.7830731929112658, 'f1-score': 0.7949719375455753, 'support': 29841.0} | {'precision': 0.8389379916879856, 'recall': 0.8401192989511075, 'f1-score': 0.8390140076168712, 'support': 29841.0} |
80
+ | No log | 8.0 | 328 | 0.5759 | {'precision': 0.599912453490917, 'recall': 0.6014922097871407, 'f1-score': 0.6007012930089853, 'support': 4557.0} | {'precision': 0.8645575877409788, 'recall': 0.7708241516086382, 'f1-score': 0.8150046598322461, 'support': 2269.0} | {'precision': 0.9148230088495575, 'recall': 0.8776087725504068, 'f1-score': 0.8958295721249322, 'support': 8481.0} | {'precision': 0.8694501422616291, 'recall': 0.9040869684876841, 'f1-score': 0.8864303302189092, 'support': 14534.0} | 0.8402 | {'precision': 0.8121857980857706, 'recall': 0.7885030256084674, 'f1-score': 0.7994914637962682, 'support': 29841.0} | {'precision': 0.8408124567818104, 'recall': 0.8402198317750745, 'f1-score': 0.8400372100799064, 'support': 29841.0} |
81
+ | No log | 9.0 | 369 | 0.5976 | {'precision': 0.6026747195858498, 'recall': 0.6131226684222076, 'f1-score': 0.6078538018057218, 'support': 4557.0} | {'precision': 0.8060156931124673, 'recall': 0.8148964301454386, 'f1-score': 0.8104317335086566, 'support': 2269.0} | {'precision': 0.9120731707317074, 'recall': 0.8818535550053059, 'f1-score': 0.8967088304058509, 'support': 8481.0} | {'precision': 0.8804975868397797, 'recall': 0.8912205862116417, 'f1-score': 0.8858266370319713, 'support': 14534.0} | 0.8403 | {'precision': 0.800315292567451, 'recall': 0.8002733099461484, 'f1-score': 0.8002052506880502, 'support': 29841.0} | {'precision': 0.8413820848138425, 'recall': 0.8402868536577193, 'f1-score': 0.8407376197665799, 'support': 29841.0} |
82
+ | No log | 10.0 | 410 | 0.6327 | {'precision': 0.6153846153846154, 'recall': 0.6179504059688391, 'f1-score': 0.6166648417825469, 'support': 4557.0} | {'precision': 0.7641955835962145, 'recall': 0.8541207580431909, 'f1-score': 0.8066597294484912, 'support': 2269.0} | {'precision': 0.9150908869098451, 'recall': 0.8844475887277443, 'f1-score': 0.8995083343326538, 'support': 8481.0} | {'precision': 0.8894852738783374, 'recall': 0.8893628732626944, 'f1-score': 0.8894240693593889, 'support': 14534.0} | 0.8438 | {'precision': 0.7960390899422531, 'recall': 0.8114704065006171, 'f1-score': 0.8030642437307701, 'support': 29841.0} | {'precision': 0.8453782465037248, 'recall': 0.8438390134378875, 'f1-score': 0.8443440976397001, 'support': 29841.0} |
83
+ | No log | 11.0 | 451 | 0.6347 | {'precision': 0.5944913550462404, 'recall': 0.6488918147904323, 'f1-score': 0.6205015213513796, 'support': 4557.0} | {'precision': 0.78500823723229, 'recall': 0.8400176289114147, 'f1-score': 0.8115818607621886, 'support': 2269.0} | {'precision': 0.9032919329555047, 'recall': 0.8832684824902723, 'f1-score': 0.8931679980922858, 'support': 8481.0} | {'precision': 0.8962250812950657, 'recall': 0.872299435805697, 'f1-score': 0.8841004184100418, 'support': 14534.0} | 0.8388 | {'precision': 0.7947541516322751, 'recall': 0.8111193404994541, 'f1-score': 0.802337949653974, 'support': 29841.0} | {'precision': 0.843699440707882, 'recall': 0.8388458831808585, 'f1-score': 0.8409094181783406, 'support': 29841.0} |
84
+ | No log | 12.0 | 492 | 0.6513 | {'precision': 0.6110076557003932, 'recall': 0.6480140443274084, 'f1-score': 0.6289669861554845, 'support': 4557.0} | {'precision': 0.803946803946804, 'recall': 0.8259144997796386, 'f1-score': 0.8147826086956521, 'support': 2269.0} | {'precision': 0.901905099988167, 'recall': 0.8987147742011555, 'f1-score': 0.9003071107961257, 'support': 8481.0} | {'precision': 0.8980036552790664, 'recall': 0.8789734415852484, 'f1-score': 0.8883866481223922, 'support': 14534.0} | 0.8453 | {'precision': 0.8037158037286076, 'recall': 0.8129041899733627, 'f1-score': 0.8081108384424137, 'support': 29841.0} | {'precision': 0.8481337577161484, 'recall': 0.8452799839147481, 'f1-score': 0.8465621274593268, 'support': 29841.0} |
85
+ | 0.2641 | 13.0 | 533 | 0.6643 | {'precision': 0.6193424423569599, 'recall': 0.6366030283080975, 'f1-score': 0.6278541283410888, 'support': 4557.0} | {'precision': 0.8395522388059702, 'recall': 0.7933010136624064, 'f1-score': 0.8157715839564923, 'support': 2269.0} | {'precision': 0.8938955172014363, 'recall': 0.9099162834571395, 'f1-score': 0.9018347551712048, 'support': 8481.0} | {'precision': 0.8938108484005564, 'recall': 0.8843401678822073, 'f1-score': 0.8890502870581725, 'support': 14534.0} | 0.8469 | {'precision': 0.8116502616912307, 'recall': 0.8060401233274627, 'f1-score': 0.8086276886317396, 'support': 29841.0} | {'precision': 0.8477953919677784, 'recall': 0.8468549981568982, 'f1-score': 0.8472247718762136, 'support': 29841.0} |
86
+ | 0.2641 | 14.0 | 574 | 0.6926 | {'precision': 0.5876997774630791, 'recall': 0.6374807987711214, 'f1-score': 0.6115789473684211, 'support': 4557.0} | {'precision': 0.8265213442325159, 'recall': 0.8021154693697664, 'f1-score': 0.814135540147618, 'support': 2269.0} | {'precision': 0.8866728153101222, 'recall': 0.9068506072397123, 'f1-score': 0.8966482075196736, 'support': 8481.0} | {'precision': 0.8985879332477535, 'recall': 0.8669327095087381, 'f1-score': 0.8824765373301583, 'support': 14534.0} | 0.8383 | {'precision': 0.7998704675633677, 'recall': 0.8033448962223346, 'f1-score': 0.8012098080914677, 'support': 29841.0} | {'precision': 0.8422463719188642, 'recall': 0.8383097081197011, 'f1-score': 0.8399392193721293, 'support': 29841.0} |
87
+ | 0.2641 | 15.0 | 615 | 0.6816 | {'precision': 0.594445578925873, 'recall': 0.6387974544656573, 'f1-score': 0.6158239898455681, 'support': 4557.0} | {'precision': 0.8150388936905791, 'recall': 0.8312031732040547, 'f1-score': 0.8230416757582371, 'support': 2269.0} | {'precision': 0.9040445973194164, 'recall': 0.8987147742011555, 'f1-score': 0.901371807000946, 'support': 8481.0} | {'precision': 0.8959785900415522, 'recall': 0.8753268198706481, 'f1-score': 0.885532314760032, 'support': 14534.0} | 0.8425 | {'precision': 0.8023769149943552, 'recall': 0.8110105554353789, 'f1-score': 0.8064424468411957, 'support': 29841.0} | {'precision': 0.8460697299178652, 'recall': 0.8424985757849938, 'f1-score': 0.8440954539700084, 'support': 29841.0} |
88
+ | 0.2641 | 16.0 | 656 | 0.6960 | {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0} | {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0} | {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0} | {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0} | 0.8446 | {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0} | {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0} |
89
 
90
 
91
  ### Framework versions
meta_data/meta_s42_e16_cvi2.json CHANGED
@@ -1 +1 @@
1
- {"Claim": {"precision": 0.594445578925873, "recall": 0.6387974544656573, "f1-score": 0.6158239898455681, "support": 4557.0}, "MajorClaim": {"precision": 0.8150388936905791, "recall": 0.8312031732040547, "f1-score": 0.8230416757582371, "support": 2269.0}, "O": {"precision": 0.9040445973194164, "recall": 0.8987147742011555, "f1-score": 0.901371807000946, "support": 8481.0}, "Premise": {"precision": 0.8959785900415522, "recall": 0.8753268198706481, "f1-score": 0.885532314760032, "support": 14534.0}, "accuracy": 0.8424985757849938, "macro avg": {"precision": 0.8023769149943552, "recall": 0.8110105554353789, "f1-score": 0.8064424468411957, "support": 29841.0}, "weighted avg": {"precision": 0.8460697299178652, "recall": 0.8424985757849938, "f1-score": 0.8440954539700084, "support": 29841.0}}
 
1
+ {"Claim": {"precision": 0.6125356125356125, "recall": 0.6133421110379635, "f1-score": 0.6129385964912281, "support": 4557.0}, "MajorClaim": {"precision": 0.8268884892086331, "recall": 0.8104892022917585, "f1-score": 0.818606721566882, "support": 2269.0}, "O": {"precision": 0.8947368421052632, "recall": 0.9040207522697795, "f1-score": 0.8993548387096775, "support": 8481.0}, "Premise": {"precision": 0.890714532274767, "recall": 0.8877115728636301, "f1-score": 0.8892105172473207, "support": 14534.0}, "accuracy": 0.844576254146979, "macro avg": {"precision": 0.806218869031069, "recall": 0.8038909096157829, "f1-score": 0.805027668503777, "support": 29841.0}, "weighted avg": {"precision": 0.8445240755442303, "recall": 0.844576254146979, "f1-score": 0.84453583593764, "support": 29841.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2a6646aa66b3a94c59ffa3908580c116e3110b578a9fc325fbdb06c314dbffa3
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01017aa6ae2032c3d14b137546353b20850c436ef6fbf493015afef9b7c7ece4
3
  size 592324828