Hugging Face's logo

Theoreticallyhugo
/
longformer-simple

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
025546c
•
1 Parent(s): 2722470

trainer: training complete at 2024-03-02 13:40:54.470673.

Browse files
Files changed (4) hide show
  1. README.md +28 -28
  2. meta_data/README_s42_e16.md +28 -28
  3. meta_data/meta_s42_e16_cvi3.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[40%:60%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.844576254146979
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6960
36
- - Claim: {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0}
37
- - Majorclaim: {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0}
38
- - O: {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0}
39
- - Premise: {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0}
40
- - Accuracy: 0.8446
41
- - Macro avg: {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0}
42
- - Weighted avg: {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.5869 | {'precision': 0.535017852238396, 'recall': 0.4274742154926487, 'f1-score': 0.47523786289338865, 'support': 4557.0} | {'precision': 0.5474121647147714, 'recall': 0.6386073159982371, 'f1-score': 0.5895036615134255, 'support': 2269.0} | {'precision': 0.845403060609712, 'recall': 0.8272609362103526, 'f1-score': 0.8362336114421931, 'support': 8481.0} | {'precision': 0.850072112232857, 'recall': 0.8921838447777625, 'f1-score': 0.8706190412246543, 'support': 14534.0} | 0.7835 | {'precision': 0.694476297448934, 'recall': 0.6963815781197502, 'f1-score': 0.6928985442684154, 'support': 29841.0} | {'precision': 0.7776202536983177, 'recall': 0.7834858081163499, 'f1-score': 0.7790930985214805, 'support': 29841.0} |
74
- | No log | 2.0 | 82 | 0.4861 | {'precision': 0.6276500447894894, 'recall': 0.4612683783190696, 'f1-score': 0.5317480394636985, 'support': 4557.0} | {'precision': 0.6855268552685527, 'recall': 0.7368884971353019, 'f1-score': 0.7102803738317758, 'support': 2269.0} | {'precision': 0.872397977184523, 'recall': 0.8746610069567268, 'f1-score': 0.8735280263777673, 'support': 8481.0} | {'precision': 0.8556913183279743, 'recall': 0.9155084629145452, 'f1-score': 0.884589815184151, 'support': 14534.0} | 0.8210 | {'precision': 0.7603165488926349, 'recall': 0.7470815863314109, 'f1-score': 0.7500365637143481, 'support': 29841.0} | {'precision': 0.8126767385071133, 'recall': 0.820951040514728, 'f1-score': 0.8143098940939201, 'support': 29841.0} |
75
- | No log | 3.0 | 123 | 0.4651 | {'precision': 0.5519389190275267, 'recall': 0.6028088654816766, 'f1-score': 0.5762534088525278, 'support': 4557.0} | {'precision': 0.6811542572141076, 'recall': 0.8426619656236227, 'f1-score': 0.7533490937746257, 'support': 2269.0} | {'precision': 0.9045047256658892, 'recall': 0.868883386393114, 'f1-score': 0.8863363002165023, 'support': 8481.0} | {'precision': 0.8910855499640546, 'recall': 0.8528278519333975, 'f1-score': 0.8715370552664885, 'support': 14534.0} | 0.8184 | {'precision': 0.7571708629678946, 'recall': 0.7917955173579527, 'f1-score': 0.7718689645275361, 'support': 29841.0} | {'precision': 0.8271460951435015, 'recall': 0.8184377199155525, 'f1-score': 0.8216639389194362, 'support': 29841.0} |
76
- | No log | 4.0 | 164 | 0.4685 | {'precision': 0.5727291118753793, 'recall': 0.6212420452051789, 'f1-score': 0.5960000000000001, 'support': 4557.0} | {'precision': 0.7450166112956811, 'recall': 0.7906566769501984, 'f1-score': 0.7671584348941629, 'support': 2269.0} | {'precision': 0.8872651356993737, 'recall': 0.9020162716660771, 'f1-score': 0.8945798982634625, 'support': 8481.0} | {'precision': 0.8981107585809057, 'recall': 0.8569561029310582, 'f1-score': 0.8770509119076122, 'support': 14534.0} | 0.8287 | {'precision': 0.7757804043628349, 'recall': 0.792717774188128, 'f1-score': 0.7836973112663095, 'support': 29841.0} | {'precision': 0.8336988249364055, 'recall': 0.8287255789015113, 'f1-score': 0.8307578351802057, 'support': 29841.0} |
77
- | No log | 5.0 | 205 | 0.4714 | {'precision': 0.6111239326102008, 'recall': 0.5810840465218345, 'f1-score': 0.5957255343082115, 'support': 4557.0} | {'precision': 0.7859340659340659, 'recall': 0.7880123402379903, 'f1-score': 0.7869718309859155, 'support': 2269.0} | {'precision': 0.8915166490175315, 'recall': 0.8934087961325315, 'f1-score': 0.8924617196702003, 'support': 8481.0} | {'precision': 0.8798018189222208, 'recall': 0.8919086280445852, 'f1-score': 0.8858138581385815, 'support': 14534.0} | 0.8370 | {'precision': 0.7920941166210047, 'recall': 0.7886034527342354, 'f1-score': 0.7902432357757272, 'support': 29841.0} | {'precision': 0.8349642603479214, 'recall': 0.8369692704668074, 'f1-score': 0.8358884354766487, 'support': 29841.0} |
78
- | No log | 6.0 | 246 | 0.5037 | {'precision': 0.5850368809272919, 'recall': 0.6091727013385999, 'f1-score': 0.5968608901311546, 'support': 4557.0} | {'precision': 0.8594594594594595, 'recall': 0.7708241516086382, 'f1-score': 0.8127323420074349, 'support': 2269.0} | {'precision': 0.8999051233396584, 'recall': 0.8947058129937507, 'f1-score': 0.8972979364985514, 'support': 8481.0} | {'precision': 0.8796910246770114, 'recall': 0.8854410348149168, 'f1-score': 0.8825566642663649, 'support': 14534.0} | 0.8372 | {'precision': 0.8060231221008552, 'recall': 0.7900359251889764, 'f1-score': 0.7973619582258764, 'support': 29841.0} | {'precision': 0.8389012192486348, 'recall': 0.8371703361147415, 'f1-score': 0.8378086229762441, 'support': 29841.0} |
79
- | No log | 7.0 | 287 | 0.5330 | {'precision': 0.6130337078651685, 'recall': 0.5986394557823129, 'f1-score': 0.6057510824913955, 'support': 4557.0} | {'precision': 0.8630921395106715, 'recall': 0.7307183781401498, 'f1-score': 0.7914081145584726, 'support': 2269.0} | {'precision': 0.8824737562756733, 'recall': 0.9119207640608419, 'f1-score': 0.8969556393157437, 'support': 8481.0} | {'precision': 0.880592955256358, 'recall': 0.8910141736617586, 'f1-score': 0.8857729138166894, 'support': 14534.0} | 0.8401 | {'precision': 0.8097981397269679, 'recall': 0.7830731929112658, 'f1-score': 0.7949719375455753, 'support': 29841.0} | {'precision': 0.8389379916879856, 'recall': 0.8401192989511075, 'f1-score': 0.8390140076168712, 'support': 29841.0} |
80
- | No log | 8.0 | 328 | 0.5759 | {'precision': 0.599912453490917, 'recall': 0.6014922097871407, 'f1-score': 0.6007012930089853, 'support': 4557.0} | {'precision': 0.8645575877409788, 'recall': 0.7708241516086382, 'f1-score': 0.8150046598322461, 'support': 2269.0} | {'precision': 0.9148230088495575, 'recall': 0.8776087725504068, 'f1-score': 0.8958295721249322, 'support': 8481.0} | {'precision': 0.8694501422616291, 'recall': 0.9040869684876841, 'f1-score': 0.8864303302189092, 'support': 14534.0} | 0.8402 | {'precision': 0.8121857980857706, 'recall': 0.7885030256084674, 'f1-score': 0.7994914637962682, 'support': 29841.0} | {'precision': 0.8408124567818104, 'recall': 0.8402198317750745, 'f1-score': 0.8400372100799064, 'support': 29841.0} |
81
- | No log | 9.0 | 369 | 0.5976 | {'precision': 0.6026747195858498, 'recall': 0.6131226684222076, 'f1-score': 0.6078538018057218, 'support': 4557.0} | {'precision': 0.8060156931124673, 'recall': 0.8148964301454386, 'f1-score': 0.8104317335086566, 'support': 2269.0} | {'precision': 0.9120731707317074, 'recall': 0.8818535550053059, 'f1-score': 0.8967088304058509, 'support': 8481.0} | {'precision': 0.8804975868397797, 'recall': 0.8912205862116417, 'f1-score': 0.8858266370319713, 'support': 14534.0} | 0.8403 | {'precision': 0.800315292567451, 'recall': 0.8002733099461484, 'f1-score': 0.8002052506880502, 'support': 29841.0} | {'precision': 0.8413820848138425, 'recall': 0.8402868536577193, 'f1-score': 0.8407376197665799, 'support': 29841.0} |
82
- | No log | 10.0 | 410 | 0.6327 | {'precision': 0.6153846153846154, 'recall': 0.6179504059688391, 'f1-score': 0.6166648417825469, 'support': 4557.0} | {'precision': 0.7641955835962145, 'recall': 0.8541207580431909, 'f1-score': 0.8066597294484912, 'support': 2269.0} | {'precision': 0.9150908869098451, 'recall': 0.8844475887277443, 'f1-score': 0.8995083343326538, 'support': 8481.0} | {'precision': 0.8894852738783374, 'recall': 0.8893628732626944, 'f1-score': 0.8894240693593889, 'support': 14534.0} | 0.8438 | {'precision': 0.7960390899422531, 'recall': 0.8114704065006171, 'f1-score': 0.8030642437307701, 'support': 29841.0} | {'precision': 0.8453782465037248, 'recall': 0.8438390134378875, 'f1-score': 0.8443440976397001, 'support': 29841.0} |
83
- | No log | 11.0 | 451 | 0.6347 | {'precision': 0.5944913550462404, 'recall': 0.6488918147904323, 'f1-score': 0.6205015213513796, 'support': 4557.0} | {'precision': 0.78500823723229, 'recall': 0.8400176289114147, 'f1-score': 0.8115818607621886, 'support': 2269.0} | {'precision': 0.9032919329555047, 'recall': 0.8832684824902723, 'f1-score': 0.8931679980922858, 'support': 8481.0} | {'precision': 0.8962250812950657, 'recall': 0.872299435805697, 'f1-score': 0.8841004184100418, 'support': 14534.0} | 0.8388 | {'precision': 0.7947541516322751, 'recall': 0.8111193404994541, 'f1-score': 0.802337949653974, 'support': 29841.0} | {'precision': 0.843699440707882, 'recall': 0.8388458831808585, 'f1-score': 0.8409094181783406, 'support': 29841.0} |
84
- | No log | 12.0 | 492 | 0.6513 | {'precision': 0.6110076557003932, 'recall': 0.6480140443274084, 'f1-score': 0.6289669861554845, 'support': 4557.0} | {'precision': 0.803946803946804, 'recall': 0.8259144997796386, 'f1-score': 0.8147826086956521, 'support': 2269.0} | {'precision': 0.901905099988167, 'recall': 0.8987147742011555, 'f1-score': 0.9003071107961257, 'support': 8481.0} | {'precision': 0.8980036552790664, 'recall': 0.8789734415852484, 'f1-score': 0.8883866481223922, 'support': 14534.0} | 0.8453 | {'precision': 0.8037158037286076, 'recall': 0.8129041899733627, 'f1-score': 0.8081108384424137, 'support': 29841.0} | {'precision': 0.8481337577161484, 'recall': 0.8452799839147481, 'f1-score': 0.8465621274593268, 'support': 29841.0} |
85
- | 0.2641 | 13.0 | 533 | 0.6643 | {'precision': 0.6193424423569599, 'recall': 0.6366030283080975, 'f1-score': 0.6278541283410888, 'support': 4557.0} | {'precision': 0.8395522388059702, 'recall': 0.7933010136624064, 'f1-score': 0.8157715839564923, 'support': 2269.0} | {'precision': 0.8938955172014363, 'recall': 0.9099162834571395, 'f1-score': 0.9018347551712048, 'support': 8481.0} | {'precision': 0.8938108484005564, 'recall': 0.8843401678822073, 'f1-score': 0.8890502870581725, 'support': 14534.0} | 0.8469 | {'precision': 0.8116502616912307, 'recall': 0.8060401233274627, 'f1-score': 0.8086276886317396, 'support': 29841.0} | {'precision': 0.8477953919677784, 'recall': 0.8468549981568982, 'f1-score': 0.8472247718762136, 'support': 29841.0} |
86
- | 0.2641 | 14.0 | 574 | 0.6926 | {'precision': 0.5876997774630791, 'recall': 0.6374807987711214, 'f1-score': 0.6115789473684211, 'support': 4557.0} | {'precision': 0.8265213442325159, 'recall': 0.8021154693697664, 'f1-score': 0.814135540147618, 'support': 2269.0} | {'precision': 0.8866728153101222, 'recall': 0.9068506072397123, 'f1-score': 0.8966482075196736, 'support': 8481.0} | {'precision': 0.8985879332477535, 'recall': 0.8669327095087381, 'f1-score': 0.8824765373301583, 'support': 14534.0} | 0.8383 | {'precision': 0.7998704675633677, 'recall': 0.8033448962223346, 'f1-score': 0.8012098080914677, 'support': 29841.0} | {'precision': 0.8422463719188642, 'recall': 0.8383097081197011, 'f1-score': 0.8399392193721293, 'support': 29841.0} |
87
- | 0.2641 | 15.0 | 615 | 0.6816 | {'precision': 0.594445578925873, 'recall': 0.6387974544656573, 'f1-score': 0.6158239898455681, 'support': 4557.0} | {'precision': 0.8150388936905791, 'recall': 0.8312031732040547, 'f1-score': 0.8230416757582371, 'support': 2269.0} | {'precision': 0.9040445973194164, 'recall': 0.8987147742011555, 'f1-score': 0.901371807000946, 'support': 8481.0} | {'precision': 0.8959785900415522, 'recall': 0.8753268198706481, 'f1-score': 0.885532314760032, 'support': 14534.0} | 0.8425 | {'precision': 0.8023769149943552, 'recall': 0.8110105554353789, 'f1-score': 0.8064424468411957, 'support': 29841.0} | {'precision': 0.8460697299178652, 'recall': 0.8424985757849938, 'f1-score': 0.8440954539700084, 'support': 29841.0} |
88
- | 0.2641 | 16.0 | 656 | 0.6960 | {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0} | {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0} | {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0} | {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0} | 0.8446 | {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0} | {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[60%:80%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.858776119402985
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6472
36
+ - Claim: {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0}
37
+ - Majorclaim: {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0}
38
+ - O: {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0}
39
+ - Premise: {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0}
40
+ - Accuracy: 0.8588
41
+ - Macro avg: {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0}
42
+ - Weighted avg: {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.6237 | {'precision': 0.4813399941228328, 'recall': 0.33157894736842103, 'f1-score': 0.3926645091693635, 'support': 4940.0} | {'precision': 0.41758530183727033, 'recall': 0.7271480804387569, 'f1-score': 0.5305101700566855, 'support': 2188.0} | {'precision': 0.8614998552263295, 'recall': 0.8522868328081734, 'f1-score': 0.8568685802054334, 'support': 10473.0} | {'precision': 0.8528192892126083, 'recall': 0.8542675639977357, 'f1-score': 0.8535428122545169, 'support': 15899.0} | 0.7683 | {'precision': 0.6533111100997602, 'recall': 0.6913203561532717, 'f1-score': 0.6583965179214999, 'support': 33500.0} | {'precision': 0.7723271066974134, 'recall': 0.7682686567164179, 'f1-score': 0.7655218131315448, 'support': 33500.0} |
74
+ | No log | 2.0 | 82 | 0.4751 | {'precision': 0.5846230654018971, 'recall': 0.47408906882591095, 'f1-score': 0.5235859602056785, 'support': 4940.0} | {'precision': 0.7269767441860465, 'recall': 0.7143510054844607, 'f1-score': 0.7206085753803596, 'support': 2188.0} | {'precision': 0.9142337609859582, 'recall': 0.8641268022534135, 'f1-score': 0.8884743765953269, 'support': 10473.0} | {'precision': 0.8357695614789338, 'recall': 0.917038807472168, 'f1-score': 0.8745201535508637, 'support': 15899.0} | 0.8219 | {'precision': 0.7654007830132088, 'recall': 0.7424014210089883, 'f1-score': 0.7517972664330571, 'support': 33500.0} | {'precision': 0.816159208839521, 'recall': 0.8219402985074626, 'f1-score': 0.8170804260816812, 'support': 33500.0} |
75
+ | No log | 3.0 | 123 | 0.4586 | {'precision': 0.6658894070619586, 'recall': 0.4046558704453441, 'f1-score': 0.5033996474439688, 'support': 4940.0} | {'precision': 0.7872244714349977, 'recall': 0.79981718464351, 'f1-score': 0.7934708682838358, 'support': 2188.0} | {'precision': 0.9342819121711536, 'recall': 0.8714790413444095, 'f1-score': 0.9017883608339096, 'support': 10473.0} | {'precision': 0.8168702042580784, 'recall': 0.9508145166362665, 'f1-score': 0.8787676209853219, 'support': 15899.0} | 0.8356 | {'precision': 0.8010664987315472, 'recall': 0.7566916532673825, 'f1-score': 0.7693566243867591, 'support': 33500.0} | {'precision': 0.8293759599418965, 'recall': 0.8356119402985075, 'f1-score': 0.8250407291712659, 'support': 33500.0} |
76
+ | No log | 4.0 | 164 | 0.4525 | {'precision': 0.5575898801597869, 'recall': 0.6781376518218624, 'f1-score': 0.6119839240043845, 'support': 4940.0} | {'precision': 0.7466456195737964, 'recall': 0.8647166361974405, 'f1-score': 0.8013553578991952, 'support': 2188.0} | {'precision': 0.9201592832254853, 'recall': 0.8825551417931825, 'f1-score': 0.9009650063359004, 'support': 10473.0} | {'precision': 0.8922416683430564, 'recall': 0.836907981634065, 'f1-score': 0.8636894716344281, 'support': 15899.0} | 0.8296 | {'precision': 0.7791591128255312, 'recall': 0.8155793528616375, 'f1-score': 0.7944984399684771, 'support': 33500.0} | {'precision': 0.8421114352783158, 'recall': 0.8295820895522388, 'f1-score': 0.8341543739861718, 'support': 33500.0} |
77
+ | No log | 5.0 | 205 | 0.4721 | {'precision': 0.662877030162413, 'recall': 0.5783400809716599, 'f1-score': 0.6177297297297297, 'support': 4940.0} | {'precision': 0.7945205479452054, 'recall': 0.8747714808043876, 'f1-score': 0.8327169893408746, 'support': 2188.0} | {'precision': 0.9125229313507772, 'recall': 0.9024157357013273, 'f1-score': 0.9074411905904946, 'support': 10473.0} | {'precision': 0.8726254262055528, 'recall': 0.9014403421598842, 'f1-score': 0.8867988738669058, 'support': 15899.0} | 0.8524 | {'precision': 0.8106364839159872, 'recall': 0.8142419099093148, 'f1-score': 0.8111716958820011, 'support': 33500.0} | {'precision': 0.8490670984831405, 'recall': 0.8523582089552239, 'f1-score': 0.8500422842449816, 'support': 33500.0} |
78
+ | No log | 6.0 | 246 | 0.4792 | {'precision': 0.6428419936373276, 'recall': 0.6135627530364373, 'f1-score': 0.6278612118073537, 'support': 4940.0} | {'precision': 0.804950917626974, 'recall': 0.8619744058500914, 'f1-score': 0.83248730964467, 'support': 2188.0} | {'precision': 0.9285714285714286, 'recall': 0.8949680129857729, 'f1-score': 0.9114601059950406, 'support': 10473.0} | {'precision': 0.872155615365794, 'recall': 0.8967859613812189, 'f1-score': 0.8842993146649301, 'support': 15899.0} | 0.8522 | {'precision': 0.812129988800381, 'recall': 0.8168227833133802, 'f1-score': 0.8140269855279986, 'support': 33500.0} | {'precision': 0.8515881419840463, 'recall': 0.8521791044776119, 'f1-score': 0.8515914362320791, 'support': 33500.0} |
79
+ | No log | 7.0 | 287 | 0.5202 | {'precision': 0.6744186046511628, 'recall': 0.5342105263157895, 'f1-score': 0.5961820851688694, 'support': 4940.0} | {'precision': 0.8121475054229935, 'recall': 0.8555758683729433, 'f1-score': 0.8332962385933673, 'support': 2188.0} | {'precision': 0.9198786930150655, 'recall': 0.8978325217225246, 'f1-score': 0.9087219135056778, 'support': 10473.0} | {'precision': 0.8582063305978898, 'recall': 0.9208755267626895, 'f1-score': 0.8884371491853515, 'support': 15899.0} | 0.8524 | {'precision': 0.816162783421778, 'recall': 0.8021236107934867, 'f1-score': 0.8066593466133165, 'support': 33500.0} | {'precision': 0.8473766761482054, 'recall': 0.8523880597014926, 'f1-score': 0.8480805524125185, 'support': 33500.0} |
80
+ | No log | 8.0 | 328 | 0.5458 | {'precision': 0.6705622932745314, 'recall': 0.6155870445344129, 'f1-score': 0.6418997361477573, 'support': 4940.0} | {'precision': 0.8129251700680272, 'recall': 0.8738574040219378, 'f1-score': 0.8422907488986784, 'support': 2188.0} | {'precision': 0.9259259259259259, 'recall': 0.89277188962093, 'f1-score': 0.909046716251033, 'support': 10473.0} | {'precision': 0.8728428701180745, 'recall': 0.9066607962764954, 'f1-score': 0.8894304929968533, 'support': 15899.0} | 0.8573 | {'precision': 0.8205640648466398, 'recall': 0.822219283613444, 'f1-score': 0.8206669235735804, 'support': 33500.0} | {'precision': 0.8556957914959558, 'recall': 0.8572537313432835, 'f1-score': 0.8559826424660975, 'support': 33500.0} |
81
+ | No log | 9.0 | 369 | 0.5550 | {'precision': 0.6423661737138097, 'recall': 0.6242914979757085, 'f1-score': 0.6331998768093625, 'support': 4940.0} | {'precision': 0.8291592128801432, 'recall': 0.8473491773308958, 'f1-score': 0.8381555153707052, 'support': 2188.0} | {'precision': 0.909720885466795, 'recall': 0.9025112193258856, 'f1-score': 0.9061017111633034, 'support': 10473.0} | {'precision': 0.8796739874323399, 'recall': 0.8893012139128247, 'f1-score': 0.8844614037282621, 'support': 15899.0} | 0.8516 | {'precision': 0.8152300648732719, 'recall': 0.8158632771363286, 'f1-score': 0.8154796267679083, 'support': 33500.0} | {'precision': 0.8507741138987609, 'recall': 0.8516119402985075, 'f1-score': 0.8511506488942767, 'support': 33500.0} |
82
+ | No log | 10.0 | 410 | 0.5788 | {'precision': 0.6611198560827524, 'recall': 0.5951417004048583, 'f1-score': 0.6263982102908278, 'support': 4940.0} | {'precision': 0.8315460232350312, 'recall': 0.8505484460694699, 'f1-score': 0.8409399005874378, 'support': 2188.0} | {'precision': 0.9248446592366111, 'recall': 0.8953499474840065, 'f1-score': 0.9098583349505143, 'support': 10473.0} | {'precision': 0.8645358599184456, 'recall': 0.9067865903515945, 'f1-score': 0.8851573292402148, 'support': 15899.0} | 0.8536 | {'precision': 0.8205115996182102, 'recall': 0.8119566710774824, 'f1-score': 0.8155884437672487, 'support': 33500.0} | {'precision': 0.851239060922849, 'recall': 0.8535820895522388, 'f1-score': 0.8518342203238483, 'support': 33500.0} |
83
+ | No log | 11.0 | 451 | 0.5865 | {'precision': 0.661878453038674, 'recall': 0.6062753036437247, 'f1-score': 0.6328578975171685, 'support': 4940.0} | {'precision': 0.829535495179667, 'recall': 0.8651736745886655, 'f1-score': 0.8469798657718122, 'support': 2188.0} | {'precision': 0.9291244788564622, 'recall': 0.8937267258665139, 'f1-score': 0.9110819097678493, 'support': 10473.0} | {'precision': 0.8703893134364282, 'recall': 0.9098056481539719, 'f1-score': 0.88966111076942, 'support': 15899.0} | 0.8571 | {'precision': 0.8227319351278078, 'recall': 0.818745338063219, 'f1-score': 0.8201451959565625, 'support': 33500.0} | {'precision': 0.8553356293389153, 'recall': 0.8571044776119403, 'f1-score': 0.8557012776467233, 'support': 33500.0} |
84
+ | No log | 12.0 | 492 | 0.6140 | {'precision': 0.6268885064065787, 'recall': 0.6635627530364372, 'f1-score': 0.6447044940505456, 'support': 4940.0} | {'precision': 0.8325078793336335, 'recall': 0.8450639853747715, 'f1-score': 0.8387389430709912, 'support': 2188.0} | {'precision': 0.923546196989078, 'recall': 0.896209300105032, 'f1-score': 0.9096724171351037, 'support': 10473.0} | {'precision': 0.885440926543715, 'recall': 0.8847726272092584, 'f1-score': 0.885106650726735, 'support': 15899.0} | 0.8531 | {'precision': 0.8170958773182513, 'recall': 0.8224021664313748, 'f1-score': 0.8195556262458439, 'support': 33500.0} | {'precision': 0.8557695842930038, 'recall': 0.8531343283582089, 'f1-score': 0.8543077872420695, 'support': 33500.0} |
85
+ | 0.2701 | 13.0 | 533 | 0.6368 | {'precision': 0.6831773567678612, 'recall': 0.6058704453441296, 'f1-score': 0.642205771912885, 'support': 4940.0} | {'precision': 0.8174536256323778, 'recall': 0.8861974405850092, 'f1-score': 0.8504385964912281, 'support': 2188.0} | {'precision': 0.9274289099526066, 'recall': 0.8968776854769407, 'f1-score': 0.9118974807048201, 'support': 10473.0} | {'precision': 0.8733377459534268, 'recall': 0.912887602993899, 'f1-score': 0.892674826250077, 'support': 15899.0} | 0.8609 | {'precision': 0.8253494095765681, 'recall': 0.8254582935999946, 'f1-score': 0.8243041688397525, 'support': 33500.0} | {'precision': 0.8585565514078823, 'recall': 0.8608656716417911, 'f1-score': 0.8589909116520602, 'support': 33500.0} |
86
+ | 0.2701 | 14.0 | 574 | 0.6486 | {'precision': 0.6641386782231853, 'recall': 0.6204453441295547, 'f1-score': 0.641548927263213, 'support': 4940.0} | {'precision': 0.8142076502732241, 'recall': 0.8852833638025595, 'f1-score': 0.8482592511495511, 'support': 2188.0} | {'precision': 0.9240070782540307, 'recall': 0.897450587224291, 'f1-score': 0.9105352385565513, 'support': 10473.0} | {'precision': 0.8767601322395004, 'recall': 0.9007484747468394, 'f1-score': 0.888592436323023, 'support': 15899.0} | 0.8574 | {'precision': 0.8197783847474851, 'recall': 0.8259819424758111, 'f1-score': 0.8222339633230846, 'support': 33500.0} | {'precision': 0.8560915487238994, 'recall': 0.8573731343283582, 'f1-score': 0.8563883474835222, 'support': 33500.0} |
87
+ | 0.2701 | 15.0 | 615 | 0.6462 | {'precision': 0.6603214890016921, 'recall': 0.6319838056680162, 'f1-score': 0.6458419528340918, 'support': 4940.0} | {'precision': 0.8342832091188075, 'recall': 0.8697440585009141, 'f1-score': 0.8516446632356232, 'support': 2188.0} | {'precision': 0.9237646134197859, 'recall': 0.8978325217225246, 'f1-score': 0.9106139841177611, 'support': 10473.0} | {'precision': 0.8785556645414418, 'recall': 0.9013774451223348, 'f1-score': 0.8898202477414549, 'support': 15899.0} | 0.8585 | {'precision': 0.8242312440204318, 'recall': 0.8252344577534474, 'f1-score': 0.8244802119822328, 'support': 33500.0} | {'precision': 0.8576162126600033, 'recall': 0.8584776119402985, 'f1-score': 0.8578498550646764, 'support': 33500.0} |
88
+ | 0.2701 | 16.0 | 656 | 0.6472 | {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0} | {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0} | {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0} | {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0} | 0.8588 | {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0} | {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0} |
89
 
90
 
91
  ### Framework versions
meta_data/README_s42_e16.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[40%:60%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.844576254146979
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6960
36
- - Claim: {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0}
37
- - Majorclaim: {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0}
38
- - O: {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0}
39
- - Premise: {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0}
40
- - Accuracy: 0.8446
41
- - Macro avg: {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0}
42
- - Weighted avg: {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.5869 | {'precision': 0.535017852238396, 'recall': 0.4274742154926487, 'f1-score': 0.47523786289338865, 'support': 4557.0} | {'precision': 0.5474121647147714, 'recall': 0.6386073159982371, 'f1-score': 0.5895036615134255, 'support': 2269.0} | {'precision': 0.845403060609712, 'recall': 0.8272609362103526, 'f1-score': 0.8362336114421931, 'support': 8481.0} | {'precision': 0.850072112232857, 'recall': 0.8921838447777625, 'f1-score': 0.8706190412246543, 'support': 14534.0} | 0.7835 | {'precision': 0.694476297448934, 'recall': 0.6963815781197502, 'f1-score': 0.6928985442684154, 'support': 29841.0} | {'precision': 0.7776202536983177, 'recall': 0.7834858081163499, 'f1-score': 0.7790930985214805, 'support': 29841.0} |
74
- | No log | 2.0 | 82 | 0.4861 | {'precision': 0.6276500447894894, 'recall': 0.4612683783190696, 'f1-score': 0.5317480394636985, 'support': 4557.0} | {'precision': 0.6855268552685527, 'recall': 0.7368884971353019, 'f1-score': 0.7102803738317758, 'support': 2269.0} | {'precision': 0.872397977184523, 'recall': 0.8746610069567268, 'f1-score': 0.8735280263777673, 'support': 8481.0} | {'precision': 0.8556913183279743, 'recall': 0.9155084629145452, 'f1-score': 0.884589815184151, 'support': 14534.0} | 0.8210 | {'precision': 0.7603165488926349, 'recall': 0.7470815863314109, 'f1-score': 0.7500365637143481, 'support': 29841.0} | {'precision': 0.8126767385071133, 'recall': 0.820951040514728, 'f1-score': 0.8143098940939201, 'support': 29841.0} |
75
- | No log | 3.0 | 123 | 0.4651 | {'precision': 0.5519389190275267, 'recall': 0.6028088654816766, 'f1-score': 0.5762534088525278, 'support': 4557.0} | {'precision': 0.6811542572141076, 'recall': 0.8426619656236227, 'f1-score': 0.7533490937746257, 'support': 2269.0} | {'precision': 0.9045047256658892, 'recall': 0.868883386393114, 'f1-score': 0.8863363002165023, 'support': 8481.0} | {'precision': 0.8910855499640546, 'recall': 0.8528278519333975, 'f1-score': 0.8715370552664885, 'support': 14534.0} | 0.8184 | {'precision': 0.7571708629678946, 'recall': 0.7917955173579527, 'f1-score': 0.7718689645275361, 'support': 29841.0} | {'precision': 0.8271460951435015, 'recall': 0.8184377199155525, 'f1-score': 0.8216639389194362, 'support': 29841.0} |
76
- | No log | 4.0 | 164 | 0.4685 | {'precision': 0.5727291118753793, 'recall': 0.6212420452051789, 'f1-score': 0.5960000000000001, 'support': 4557.0} | {'precision': 0.7450166112956811, 'recall': 0.7906566769501984, 'f1-score': 0.7671584348941629, 'support': 2269.0} | {'precision': 0.8872651356993737, 'recall': 0.9020162716660771, 'f1-score': 0.8945798982634625, 'support': 8481.0} | {'precision': 0.8981107585809057, 'recall': 0.8569561029310582, 'f1-score': 0.8770509119076122, 'support': 14534.0} | 0.8287 | {'precision': 0.7757804043628349, 'recall': 0.792717774188128, 'f1-score': 0.7836973112663095, 'support': 29841.0} | {'precision': 0.8336988249364055, 'recall': 0.8287255789015113, 'f1-score': 0.8307578351802057, 'support': 29841.0} |
77
- | No log | 5.0 | 205 | 0.4714 | {'precision': 0.6111239326102008, 'recall': 0.5810840465218345, 'f1-score': 0.5957255343082115, 'support': 4557.0} | {'precision': 0.7859340659340659, 'recall': 0.7880123402379903, 'f1-score': 0.7869718309859155, 'support': 2269.0} | {'precision': 0.8915166490175315, 'recall': 0.8934087961325315, 'f1-score': 0.8924617196702003, 'support': 8481.0} | {'precision': 0.8798018189222208, 'recall': 0.8919086280445852, 'f1-score': 0.8858138581385815, 'support': 14534.0} | 0.8370 | {'precision': 0.7920941166210047, 'recall': 0.7886034527342354, 'f1-score': 0.7902432357757272, 'support': 29841.0} | {'precision': 0.8349642603479214, 'recall': 0.8369692704668074, 'f1-score': 0.8358884354766487, 'support': 29841.0} |
78
- | No log | 6.0 | 246 | 0.5037 | {'precision': 0.5850368809272919, 'recall': 0.6091727013385999, 'f1-score': 0.5968608901311546, 'support': 4557.0} | {'precision': 0.8594594594594595, 'recall': 0.7708241516086382, 'f1-score': 0.8127323420074349, 'support': 2269.0} | {'precision': 0.8999051233396584, 'recall': 0.8947058129937507, 'f1-score': 0.8972979364985514, 'support': 8481.0} | {'precision': 0.8796910246770114, 'recall': 0.8854410348149168, 'f1-score': 0.8825566642663649, 'support': 14534.0} | 0.8372 | {'precision': 0.8060231221008552, 'recall': 0.7900359251889764, 'f1-score': 0.7973619582258764, 'support': 29841.0} | {'precision': 0.8389012192486348, 'recall': 0.8371703361147415, 'f1-score': 0.8378086229762441, 'support': 29841.0} |
79
- | No log | 7.0 | 287 | 0.5330 | {'precision': 0.6130337078651685, 'recall': 0.5986394557823129, 'f1-score': 0.6057510824913955, 'support': 4557.0} | {'precision': 0.8630921395106715, 'recall': 0.7307183781401498, 'f1-score': 0.7914081145584726, 'support': 2269.0} | {'precision': 0.8824737562756733, 'recall': 0.9119207640608419, 'f1-score': 0.8969556393157437, 'support': 8481.0} | {'precision': 0.880592955256358, 'recall': 0.8910141736617586, 'f1-score': 0.8857729138166894, 'support': 14534.0} | 0.8401 | {'precision': 0.8097981397269679, 'recall': 0.7830731929112658, 'f1-score': 0.7949719375455753, 'support': 29841.0} | {'precision': 0.8389379916879856, 'recall': 0.8401192989511075, 'f1-score': 0.8390140076168712, 'support': 29841.0} |
80
- | No log | 8.0 | 328 | 0.5759 | {'precision': 0.599912453490917, 'recall': 0.6014922097871407, 'f1-score': 0.6007012930089853, 'support': 4557.0} | {'precision': 0.8645575877409788, 'recall': 0.7708241516086382, 'f1-score': 0.8150046598322461, 'support': 2269.0} | {'precision': 0.9148230088495575, 'recall': 0.8776087725504068, 'f1-score': 0.8958295721249322, 'support': 8481.0} | {'precision': 0.8694501422616291, 'recall': 0.9040869684876841, 'f1-score': 0.8864303302189092, 'support': 14534.0} | 0.8402 | {'precision': 0.8121857980857706, 'recall': 0.7885030256084674, 'f1-score': 0.7994914637962682, 'support': 29841.0} | {'precision': 0.8408124567818104, 'recall': 0.8402198317750745, 'f1-score': 0.8400372100799064, 'support': 29841.0} |
81
- | No log | 9.0 | 369 | 0.5976 | {'precision': 0.6026747195858498, 'recall': 0.6131226684222076, 'f1-score': 0.6078538018057218, 'support': 4557.0} | {'precision': 0.8060156931124673, 'recall': 0.8148964301454386, 'f1-score': 0.8104317335086566, 'support': 2269.0} | {'precision': 0.9120731707317074, 'recall': 0.8818535550053059, 'f1-score': 0.8967088304058509, 'support': 8481.0} | {'precision': 0.8804975868397797, 'recall': 0.8912205862116417, 'f1-score': 0.8858266370319713, 'support': 14534.0} | 0.8403 | {'precision': 0.800315292567451, 'recall': 0.8002733099461484, 'f1-score': 0.8002052506880502, 'support': 29841.0} | {'precision': 0.8413820848138425, 'recall': 0.8402868536577193, 'f1-score': 0.8407376197665799, 'support': 29841.0} |
82
- | No log | 10.0 | 410 | 0.6327 | {'precision': 0.6153846153846154, 'recall': 0.6179504059688391, 'f1-score': 0.6166648417825469, 'support': 4557.0} | {'precision': 0.7641955835962145, 'recall': 0.8541207580431909, 'f1-score': 0.8066597294484912, 'support': 2269.0} | {'precision': 0.9150908869098451, 'recall': 0.8844475887277443, 'f1-score': 0.8995083343326538, 'support': 8481.0} | {'precision': 0.8894852738783374, 'recall': 0.8893628732626944, 'f1-score': 0.8894240693593889, 'support': 14534.0} | 0.8438 | {'precision': 0.7960390899422531, 'recall': 0.8114704065006171, 'f1-score': 0.8030642437307701, 'support': 29841.0} | {'precision': 0.8453782465037248, 'recall': 0.8438390134378875, 'f1-score': 0.8443440976397001, 'support': 29841.0} |
83
- | No log | 11.0 | 451 | 0.6347 | {'precision': 0.5944913550462404, 'recall': 0.6488918147904323, 'f1-score': 0.6205015213513796, 'support': 4557.0} | {'precision': 0.78500823723229, 'recall': 0.8400176289114147, 'f1-score': 0.8115818607621886, 'support': 2269.0} | {'precision': 0.9032919329555047, 'recall': 0.8832684824902723, 'f1-score': 0.8931679980922858, 'support': 8481.0} | {'precision': 0.8962250812950657, 'recall': 0.872299435805697, 'f1-score': 0.8841004184100418, 'support': 14534.0} | 0.8388 | {'precision': 0.7947541516322751, 'recall': 0.8111193404994541, 'f1-score': 0.802337949653974, 'support': 29841.0} | {'precision': 0.843699440707882, 'recall': 0.8388458831808585, 'f1-score': 0.8409094181783406, 'support': 29841.0} |
84
- | No log | 12.0 | 492 | 0.6513 | {'precision': 0.6110076557003932, 'recall': 0.6480140443274084, 'f1-score': 0.6289669861554845, 'support': 4557.0} | {'precision': 0.803946803946804, 'recall': 0.8259144997796386, 'f1-score': 0.8147826086956521, 'support': 2269.0} | {'precision': 0.901905099988167, 'recall': 0.8987147742011555, 'f1-score': 0.9003071107961257, 'support': 8481.0} | {'precision': 0.8980036552790664, 'recall': 0.8789734415852484, 'f1-score': 0.8883866481223922, 'support': 14534.0} | 0.8453 | {'precision': 0.8037158037286076, 'recall': 0.8129041899733627, 'f1-score': 0.8081108384424137, 'support': 29841.0} | {'precision': 0.8481337577161484, 'recall': 0.8452799839147481, 'f1-score': 0.8465621274593268, 'support': 29841.0} |
85
- | 0.2641 | 13.0 | 533 | 0.6643 | {'precision': 0.6193424423569599, 'recall': 0.6366030283080975, 'f1-score': 0.6278541283410888, 'support': 4557.0} | {'precision': 0.8395522388059702, 'recall': 0.7933010136624064, 'f1-score': 0.8157715839564923, 'support': 2269.0} | {'precision': 0.8938955172014363, 'recall': 0.9099162834571395, 'f1-score': 0.9018347551712048, 'support': 8481.0} | {'precision': 0.8938108484005564, 'recall': 0.8843401678822073, 'f1-score': 0.8890502870581725, 'support': 14534.0} | 0.8469 | {'precision': 0.8116502616912307, 'recall': 0.8060401233274627, 'f1-score': 0.8086276886317396, 'support': 29841.0} | {'precision': 0.8477953919677784, 'recall': 0.8468549981568982, 'f1-score': 0.8472247718762136, 'support': 29841.0} |
86
- | 0.2641 | 14.0 | 574 | 0.6926 | {'precision': 0.5876997774630791, 'recall': 0.6374807987711214, 'f1-score': 0.6115789473684211, 'support': 4557.0} | {'precision': 0.8265213442325159, 'recall': 0.8021154693697664, 'f1-score': 0.814135540147618, 'support': 2269.0} | {'precision': 0.8866728153101222, 'recall': 0.9068506072397123, 'f1-score': 0.8966482075196736, 'support': 8481.0} | {'precision': 0.8985879332477535, 'recall': 0.8669327095087381, 'f1-score': 0.8824765373301583, 'support': 14534.0} | 0.8383 | {'precision': 0.7998704675633677, 'recall': 0.8033448962223346, 'f1-score': 0.8012098080914677, 'support': 29841.0} | {'precision': 0.8422463719188642, 'recall': 0.8383097081197011, 'f1-score': 0.8399392193721293, 'support': 29841.0} |
87
- | 0.2641 | 15.0 | 615 | 0.6816 | {'precision': 0.594445578925873, 'recall': 0.6387974544656573, 'f1-score': 0.6158239898455681, 'support': 4557.0} | {'precision': 0.8150388936905791, 'recall': 0.8312031732040547, 'f1-score': 0.8230416757582371, 'support': 2269.0} | {'precision': 0.9040445973194164, 'recall': 0.8987147742011555, 'f1-score': 0.901371807000946, 'support': 8481.0} | {'precision': 0.8959785900415522, 'recall': 0.8753268198706481, 'f1-score': 0.885532314760032, 'support': 14534.0} | 0.8425 | {'precision': 0.8023769149943552, 'recall': 0.8110105554353789, 'f1-score': 0.8064424468411957, 'support': 29841.0} | {'precision': 0.8460697299178652, 'recall': 0.8424985757849938, 'f1-score': 0.8440954539700084, 'support': 29841.0} |
88
- | 0.2641 | 16.0 | 656 | 0.6960 | {'precision': 0.6125356125356125, 'recall': 0.6133421110379635, 'f1-score': 0.6129385964912281, 'support': 4557.0} | {'precision': 0.8268884892086331, 'recall': 0.8104892022917585, 'f1-score': 0.818606721566882, 'support': 2269.0} | {'precision': 0.8947368421052632, 'recall': 0.9040207522697795, 'f1-score': 0.8993548387096775, 'support': 8481.0} | {'precision': 0.890714532274767, 'recall': 0.8877115728636301, 'f1-score': 0.8892105172473207, 'support': 14534.0} | 0.8446 | {'precision': 0.806218869031069, 'recall': 0.8038909096157829, 'f1-score': 0.805027668503777, 'support': 29841.0} | {'precision': 0.8445240755442303, 'recall': 0.844576254146979, 'f1-score': 0.84453583593764, 'support': 29841.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[60%:80%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.858776119402985
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6472
36
+ - Claim: {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0}
37
+ - Majorclaim: {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0}
38
+ - O: {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0}
39
+ - Premise: {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0}
40
+ - Accuracy: 0.8588
41
+ - Macro avg: {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0}
42
+ - Weighted avg: {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.6237 | {'precision': 0.4813399941228328, 'recall': 0.33157894736842103, 'f1-score': 0.3926645091693635, 'support': 4940.0} | {'precision': 0.41758530183727033, 'recall': 0.7271480804387569, 'f1-score': 0.5305101700566855, 'support': 2188.0} | {'precision': 0.8614998552263295, 'recall': 0.8522868328081734, 'f1-score': 0.8568685802054334, 'support': 10473.0} | {'precision': 0.8528192892126083, 'recall': 0.8542675639977357, 'f1-score': 0.8535428122545169, 'support': 15899.0} | 0.7683 | {'precision': 0.6533111100997602, 'recall': 0.6913203561532717, 'f1-score': 0.6583965179214999, 'support': 33500.0} | {'precision': 0.7723271066974134, 'recall': 0.7682686567164179, 'f1-score': 0.7655218131315448, 'support': 33500.0} |
74
+ | No log | 2.0 | 82 | 0.4751 | {'precision': 0.5846230654018971, 'recall': 0.47408906882591095, 'f1-score': 0.5235859602056785, 'support': 4940.0} | {'precision': 0.7269767441860465, 'recall': 0.7143510054844607, 'f1-score': 0.7206085753803596, 'support': 2188.0} | {'precision': 0.9142337609859582, 'recall': 0.8641268022534135, 'f1-score': 0.8884743765953269, 'support': 10473.0} | {'precision': 0.8357695614789338, 'recall': 0.917038807472168, 'f1-score': 0.8745201535508637, 'support': 15899.0} | 0.8219 | {'precision': 0.7654007830132088, 'recall': 0.7424014210089883, 'f1-score': 0.7517972664330571, 'support': 33500.0} | {'precision': 0.816159208839521, 'recall': 0.8219402985074626, 'f1-score': 0.8170804260816812, 'support': 33500.0} |
75
+ | No log | 3.0 | 123 | 0.4586 | {'precision': 0.6658894070619586, 'recall': 0.4046558704453441, 'f1-score': 0.5033996474439688, 'support': 4940.0} | {'precision': 0.7872244714349977, 'recall': 0.79981718464351, 'f1-score': 0.7934708682838358, 'support': 2188.0} | {'precision': 0.9342819121711536, 'recall': 0.8714790413444095, 'f1-score': 0.9017883608339096, 'support': 10473.0} | {'precision': 0.8168702042580784, 'recall': 0.9508145166362665, 'f1-score': 0.8787676209853219, 'support': 15899.0} | 0.8356 | {'precision': 0.8010664987315472, 'recall': 0.7566916532673825, 'f1-score': 0.7693566243867591, 'support': 33500.0} | {'precision': 0.8293759599418965, 'recall': 0.8356119402985075, 'f1-score': 0.8250407291712659, 'support': 33500.0} |
76
+ | No log | 4.0 | 164 | 0.4525 | {'precision': 0.5575898801597869, 'recall': 0.6781376518218624, 'f1-score': 0.6119839240043845, 'support': 4940.0} | {'precision': 0.7466456195737964, 'recall': 0.8647166361974405, 'f1-score': 0.8013553578991952, 'support': 2188.0} | {'precision': 0.9201592832254853, 'recall': 0.8825551417931825, 'f1-score': 0.9009650063359004, 'support': 10473.0} | {'precision': 0.8922416683430564, 'recall': 0.836907981634065, 'f1-score': 0.8636894716344281, 'support': 15899.0} | 0.8296 | {'precision': 0.7791591128255312, 'recall': 0.8155793528616375, 'f1-score': 0.7944984399684771, 'support': 33500.0} | {'precision': 0.8421114352783158, 'recall': 0.8295820895522388, 'f1-score': 0.8341543739861718, 'support': 33500.0} |
77
+ | No log | 5.0 | 205 | 0.4721 | {'precision': 0.662877030162413, 'recall': 0.5783400809716599, 'f1-score': 0.6177297297297297, 'support': 4940.0} | {'precision': 0.7945205479452054, 'recall': 0.8747714808043876, 'f1-score': 0.8327169893408746, 'support': 2188.0} | {'precision': 0.9125229313507772, 'recall': 0.9024157357013273, 'f1-score': 0.9074411905904946, 'support': 10473.0} | {'precision': 0.8726254262055528, 'recall': 0.9014403421598842, 'f1-score': 0.8867988738669058, 'support': 15899.0} | 0.8524 | {'precision': 0.8106364839159872, 'recall': 0.8142419099093148, 'f1-score': 0.8111716958820011, 'support': 33500.0} | {'precision': 0.8490670984831405, 'recall': 0.8523582089552239, 'f1-score': 0.8500422842449816, 'support': 33500.0} |
78
+ | No log | 6.0 | 246 | 0.4792 | {'precision': 0.6428419936373276, 'recall': 0.6135627530364373, 'f1-score': 0.6278612118073537, 'support': 4940.0} | {'precision': 0.804950917626974, 'recall': 0.8619744058500914, 'f1-score': 0.83248730964467, 'support': 2188.0} | {'precision': 0.9285714285714286, 'recall': 0.8949680129857729, 'f1-score': 0.9114601059950406, 'support': 10473.0} | {'precision': 0.872155615365794, 'recall': 0.8967859613812189, 'f1-score': 0.8842993146649301, 'support': 15899.0} | 0.8522 | {'precision': 0.812129988800381, 'recall': 0.8168227833133802, 'f1-score': 0.8140269855279986, 'support': 33500.0} | {'precision': 0.8515881419840463, 'recall': 0.8521791044776119, 'f1-score': 0.8515914362320791, 'support': 33500.0} |
79
+ | No log | 7.0 | 287 | 0.5202 | {'precision': 0.6744186046511628, 'recall': 0.5342105263157895, 'f1-score': 0.5961820851688694, 'support': 4940.0} | {'precision': 0.8121475054229935, 'recall': 0.8555758683729433, 'f1-score': 0.8332962385933673, 'support': 2188.0} | {'precision': 0.9198786930150655, 'recall': 0.8978325217225246, 'f1-score': 0.9087219135056778, 'support': 10473.0} | {'precision': 0.8582063305978898, 'recall': 0.9208755267626895, 'f1-score': 0.8884371491853515, 'support': 15899.0} | 0.8524 | {'precision': 0.816162783421778, 'recall': 0.8021236107934867, 'f1-score': 0.8066593466133165, 'support': 33500.0} | {'precision': 0.8473766761482054, 'recall': 0.8523880597014926, 'f1-score': 0.8480805524125185, 'support': 33500.0} |
80
+ | No log | 8.0 | 328 | 0.5458 | {'precision': 0.6705622932745314, 'recall': 0.6155870445344129, 'f1-score': 0.6418997361477573, 'support': 4940.0} | {'precision': 0.8129251700680272, 'recall': 0.8738574040219378, 'f1-score': 0.8422907488986784, 'support': 2188.0} | {'precision': 0.9259259259259259, 'recall': 0.89277188962093, 'f1-score': 0.909046716251033, 'support': 10473.0} | {'precision': 0.8728428701180745, 'recall': 0.9066607962764954, 'f1-score': 0.8894304929968533, 'support': 15899.0} | 0.8573 | {'precision': 0.8205640648466398, 'recall': 0.822219283613444, 'f1-score': 0.8206669235735804, 'support': 33500.0} | {'precision': 0.8556957914959558, 'recall': 0.8572537313432835, 'f1-score': 0.8559826424660975, 'support': 33500.0} |
81
+ | No log | 9.0 | 369 | 0.5550 | {'precision': 0.6423661737138097, 'recall': 0.6242914979757085, 'f1-score': 0.6331998768093625, 'support': 4940.0} | {'precision': 0.8291592128801432, 'recall': 0.8473491773308958, 'f1-score': 0.8381555153707052, 'support': 2188.0} | {'precision': 0.909720885466795, 'recall': 0.9025112193258856, 'f1-score': 0.9061017111633034, 'support': 10473.0} | {'precision': 0.8796739874323399, 'recall': 0.8893012139128247, 'f1-score': 0.8844614037282621, 'support': 15899.0} | 0.8516 | {'precision': 0.8152300648732719, 'recall': 0.8158632771363286, 'f1-score': 0.8154796267679083, 'support': 33500.0} | {'precision': 0.8507741138987609, 'recall': 0.8516119402985075, 'f1-score': 0.8511506488942767, 'support': 33500.0} |
82
+ | No log | 10.0 | 410 | 0.5788 | {'precision': 0.6611198560827524, 'recall': 0.5951417004048583, 'f1-score': 0.6263982102908278, 'support': 4940.0} | {'precision': 0.8315460232350312, 'recall': 0.8505484460694699, 'f1-score': 0.8409399005874378, 'support': 2188.0} | {'precision': 0.9248446592366111, 'recall': 0.8953499474840065, 'f1-score': 0.9098583349505143, 'support': 10473.0} | {'precision': 0.8645358599184456, 'recall': 0.9067865903515945, 'f1-score': 0.8851573292402148, 'support': 15899.0} | 0.8536 | {'precision': 0.8205115996182102, 'recall': 0.8119566710774824, 'f1-score': 0.8155884437672487, 'support': 33500.0} | {'precision': 0.851239060922849, 'recall': 0.8535820895522388, 'f1-score': 0.8518342203238483, 'support': 33500.0} |
83
+ | No log | 11.0 | 451 | 0.5865 | {'precision': 0.661878453038674, 'recall': 0.6062753036437247, 'f1-score': 0.6328578975171685, 'support': 4940.0} | {'precision': 0.829535495179667, 'recall': 0.8651736745886655, 'f1-score': 0.8469798657718122, 'support': 2188.0} | {'precision': 0.9291244788564622, 'recall': 0.8937267258665139, 'f1-score': 0.9110819097678493, 'support': 10473.0} | {'precision': 0.8703893134364282, 'recall': 0.9098056481539719, 'f1-score': 0.88966111076942, 'support': 15899.0} | 0.8571 | {'precision': 0.8227319351278078, 'recall': 0.818745338063219, 'f1-score': 0.8201451959565625, 'support': 33500.0} | {'precision': 0.8553356293389153, 'recall': 0.8571044776119403, 'f1-score': 0.8557012776467233, 'support': 33500.0} |
84
+ | No log | 12.0 | 492 | 0.6140 | {'precision': 0.6268885064065787, 'recall': 0.6635627530364372, 'f1-score': 0.6447044940505456, 'support': 4940.0} | {'precision': 0.8325078793336335, 'recall': 0.8450639853747715, 'f1-score': 0.8387389430709912, 'support': 2188.0} | {'precision': 0.923546196989078, 'recall': 0.896209300105032, 'f1-score': 0.9096724171351037, 'support': 10473.0} | {'precision': 0.885440926543715, 'recall': 0.8847726272092584, 'f1-score': 0.885106650726735, 'support': 15899.0} | 0.8531 | {'precision': 0.8170958773182513, 'recall': 0.8224021664313748, 'f1-score': 0.8195556262458439, 'support': 33500.0} | {'precision': 0.8557695842930038, 'recall': 0.8531343283582089, 'f1-score': 0.8543077872420695, 'support': 33500.0} |
85
+ | 0.2701 | 13.0 | 533 | 0.6368 | {'precision': 0.6831773567678612, 'recall': 0.6058704453441296, 'f1-score': 0.642205771912885, 'support': 4940.0} | {'precision': 0.8174536256323778, 'recall': 0.8861974405850092, 'f1-score': 0.8504385964912281, 'support': 2188.0} | {'precision': 0.9274289099526066, 'recall': 0.8968776854769407, 'f1-score': 0.9118974807048201, 'support': 10473.0} | {'precision': 0.8733377459534268, 'recall': 0.912887602993899, 'f1-score': 0.892674826250077, 'support': 15899.0} | 0.8609 | {'precision': 0.8253494095765681, 'recall': 0.8254582935999946, 'f1-score': 0.8243041688397525, 'support': 33500.0} | {'precision': 0.8585565514078823, 'recall': 0.8608656716417911, 'f1-score': 0.8589909116520602, 'support': 33500.0} |
86
+ | 0.2701 | 14.0 | 574 | 0.6486 | {'precision': 0.6641386782231853, 'recall': 0.6204453441295547, 'f1-score': 0.641548927263213, 'support': 4940.0} | {'precision': 0.8142076502732241, 'recall': 0.8852833638025595, 'f1-score': 0.8482592511495511, 'support': 2188.0} | {'precision': 0.9240070782540307, 'recall': 0.897450587224291, 'f1-score': 0.9105352385565513, 'support': 10473.0} | {'precision': 0.8767601322395004, 'recall': 0.9007484747468394, 'f1-score': 0.888592436323023, 'support': 15899.0} | 0.8574 | {'precision': 0.8197783847474851, 'recall': 0.8259819424758111, 'f1-score': 0.8222339633230846, 'support': 33500.0} | {'precision': 0.8560915487238994, 'recall': 0.8573731343283582, 'f1-score': 0.8563883474835222, 'support': 33500.0} |
87
+ | 0.2701 | 15.0 | 615 | 0.6462 | {'precision': 0.6603214890016921, 'recall': 0.6319838056680162, 'f1-score': 0.6458419528340918, 'support': 4940.0} | {'precision': 0.8342832091188075, 'recall': 0.8697440585009141, 'f1-score': 0.8516446632356232, 'support': 2188.0} | {'precision': 0.9237646134197859, 'recall': 0.8978325217225246, 'f1-score': 0.9106139841177611, 'support': 10473.0} | {'precision': 0.8785556645414418, 'recall': 0.9013774451223348, 'f1-score': 0.8898202477414549, 'support': 15899.0} | 0.8585 | {'precision': 0.8242312440204318, 'recall': 0.8252344577534474, 'f1-score': 0.8244802119822328, 'support': 33500.0} | {'precision': 0.8576162126600033, 'recall': 0.8584776119402985, 'f1-score': 0.8578498550646764, 'support': 33500.0} |
88
+ | 0.2701 | 16.0 | 656 | 0.6472 | {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0} | {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0} | {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0} | {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0} | 0.8588 | {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0} | {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0} |
89
 
90
 
91
  ### Framework versions
meta_data/meta_s42_e16_cvi3.json CHANGED
@@ -1 +1 @@
1
- {"Claim": {"precision": 0.6603214890016921, "recall": 0.6319838056680162, "f1-score": 0.6458419528340918, "support": 4940.0}, "MajorClaim": {"precision": 0.8342832091188075, "recall": 0.8697440585009141, "f1-score": 0.8516446632356232, "support": 2188.0}, "O": {"precision": 0.9237646134197859, "recall": 0.8978325217225246, "f1-score": 0.9106139841177611, "support": 10473.0}, "Premise": {"precision": 0.8785556645414418, "recall": 0.9013774451223348, "f1-score": 0.8898202477414549, "support": 15899.0}, "accuracy": 0.8584776119402985, "macro avg": {"precision": 0.8242312440204318, "recall": 0.8252344577534474, "f1-score": 0.8244802119822328, "support": 33500.0}, "weighted avg": {"precision": 0.8576162126600033, "recall": 0.8584776119402985, "f1-score": 0.8578498550646764, "support": 33500.0}}
 
1
+ {"Claim": {"precision": 0.6572622779519331, "recall": 0.6366396761133604, "f1-score": 0.6467866323907456, "support": 4940.0}, "MajorClaim": {"precision": 0.8274678111587983, "recall": 0.8811700182815356, "f1-score": 0.8534749889331562, "support": 2188.0}, "O": {"precision": 0.9268028016178357, "recall": 0.8970686527260575, "f1-score": 0.9116933527413877, "support": 10473.0}, "Premise": {"precision": 0.8801698670605613, "recall": 0.8994905339958488, "f1-score": 0.8897253242915357, "support": 15899.0}, "accuracy": 0.858776119402985, "macro avg": {"precision": 0.8229256894472821, "recall": 0.8285922202792007, "f1-score": 0.8254200745892063, "support": 33500.0}, "weighted avg": {"precision": 0.8584358710936555, "recall": 0.858776119402985, "f1-score": 0.8584010941482899, "support": 33500.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc431db88957a585280816f57e4b96ae38f064c384dba8a731417633aacd382a
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4d90434bb82816205ef8b7455aa659f1e0a01a756fdd8163729ccb4e8dad86f
3
  size 592324828