Hugging Face's logo

Theoreticallyhugo
/
longformer-simple

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
972925d
•
1 Parent(s): cae9601

trainer: training complete at 2024-03-02 13:53:14.317521.

Browse files
Files changed (4) hide show
  1. README.md +28 -28
  2. meta_data/README_s42_e16.md +28 -28
  3. meta_data/meta_s42_e16_cvi4.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[60%:80%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.858776119402985
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6472
36
- - Claim: {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0}
37
- - Majorclaim: {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0}
38
- - O: {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0}
39
- - Premise: {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0}
40
- - Accuracy: 0.8588
41
- - Macro avg: {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0}
42
- - Weighted avg: {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6237 | {'precision': 0.4813399941228328, 'recall': 0.33157894736842103, 'f1-score': 0.3926645091693635, 'support': 4940.0} | {'precision': 0.41758530183727033, 'recall': 0.7271480804387569, 'f1-score': 0.5305101700566855, 'support': 2188.0} | {'precision': 0.8614998552263295, 'recall': 0.8522868328081734, 'f1-score': 0.8568685802054334, 'support': 10473.0} | {'precision': 0.8528192892126083, 'recall': 0.8542675639977357, 'f1-score': 0.8535428122545169, 'support': 15899.0} | 0.7683 | {'precision': 0.6533111100997602, 'recall': 0.6913203561532717, 'f1-score': 0.6583965179214999, 'support': 33500.0} | {'precision': 0.7723271066974134, 'recall': 0.7682686567164179, 'f1-score': 0.7655218131315448, 'support': 33500.0} |
74
- | No log | 2.0 | 82 | 0.4751 | {'precision': 0.5846230654018971, 'recall': 0.47408906882591095, 'f1-score': 0.5235859602056785, 'support': 4940.0} | {'precision': 0.7269767441860465, 'recall': 0.7143510054844607, 'f1-score': 0.7206085753803596, 'support': 2188.0} | {'precision': 0.9142337609859582, 'recall': 0.8641268022534135, 'f1-score': 0.8884743765953269, 'support': 10473.0} | {'precision': 0.8357695614789338, 'recall': 0.917038807472168, 'f1-score': 0.8745201535508637, 'support': 15899.0} | 0.8219 | {'precision': 0.7654007830132088, 'recall': 0.7424014210089883, 'f1-score': 0.7517972664330571, 'support': 33500.0} | {'precision': 0.816159208839521, 'recall': 0.8219402985074626, 'f1-score': 0.8170804260816812, 'support': 33500.0} |
75
- | No log | 3.0 | 123 | 0.4586 | {'precision': 0.6658894070619586, 'recall': 0.4046558704453441, 'f1-score': 0.5033996474439688, 'support': 4940.0} | {'precision': 0.7872244714349977, 'recall': 0.79981718464351, 'f1-score': 0.7934708682838358, 'support': 2188.0} | {'precision': 0.9342819121711536, 'recall': 0.8714790413444095, 'f1-score': 0.9017883608339096, 'support': 10473.0} | {'precision': 0.8168702042580784, 'recall': 0.9508145166362665, 'f1-score': 0.8787676209853219, 'support': 15899.0} | 0.8356 | {'precision': 0.8010664987315472, 'recall': 0.7566916532673825, 'f1-score': 0.7693566243867591, 'support': 33500.0} | {'precision': 0.8293759599418965, 'recall': 0.8356119402985075, 'f1-score': 0.8250407291712659, 'support': 33500.0} |
76
- | No log | 4.0 | 164 | 0.4525 | {'precision': 0.5575898801597869, 'recall': 0.6781376518218624, 'f1-score': 0.6119839240043845, 'support': 4940.0} | {'precision': 0.7466456195737964, 'recall': 0.8647166361974405, 'f1-score': 0.8013553578991952, 'support': 2188.0} | {'precision': 0.9201592832254853, 'recall': 0.8825551417931825, 'f1-score': 0.9009650063359004, 'support': 10473.0} | {'precision': 0.8922416683430564, 'recall': 0.836907981634065, 'f1-score': 0.8636894716344281, 'support': 15899.0} | 0.8296 | {'precision': 0.7791591128255312, 'recall': 0.8155793528616375, 'f1-score': 0.7944984399684771, 'support': 33500.0} | {'precision': 0.8421114352783158, 'recall': 0.8295820895522388, 'f1-score': 0.8341543739861718, 'support': 33500.0} |
77
- | No log | 5.0 | 205 | 0.4721 | {'precision': 0.662877030162413, 'recall': 0.5783400809716599, 'f1-score': 0.6177297297297297, 'support': 4940.0} | {'precision': 0.7945205479452054, 'recall': 0.8747714808043876, 'f1-score': 0.8327169893408746, 'support': 2188.0} | {'precision': 0.9125229313507772, 'recall': 0.9024157357013273, 'f1-score': 0.9074411905904946, 'support': 10473.0} | {'precision': 0.8726254262055528, 'recall': 0.9014403421598842, 'f1-score': 0.8867988738669058, 'support': 15899.0} | 0.8524 | {'precision': 0.8106364839159872, 'recall': 0.8142419099093148, 'f1-score': 0.8111716958820011, 'support': 33500.0} | {'precision': 0.8490670984831405, 'recall': 0.8523582089552239, 'f1-score': 0.8500422842449816, 'support': 33500.0} |
78
- | No log | 6.0 | 246 | 0.4792 | {'precision': 0.6428419936373276, 'recall': 0.6135627530364373, 'f1-score': 0.6278612118073537, 'support': 4940.0} | {'precision': 0.804950917626974, 'recall': 0.8619744058500914, 'f1-score': 0.83248730964467, 'support': 2188.0} | {'precision': 0.9285714285714286, 'recall': 0.8949680129857729, 'f1-score': 0.9114601059950406, 'support': 10473.0} | {'precision': 0.872155615365794, 'recall': 0.8967859613812189, 'f1-score': 0.8842993146649301, 'support': 15899.0} | 0.8522 | {'precision': 0.812129988800381, 'recall': 0.8168227833133802, 'f1-score': 0.8140269855279986, 'support': 33500.0} | {'precision': 0.8515881419840463, 'recall': 0.8521791044776119, 'f1-score': 0.8515914362320791, 'support': 33500.0} |
79
- | No log | 7.0 | 287 | 0.5202 | {'precision': 0.6744186046511628, 'recall': 0.5342105263157895, 'f1-score': 0.5961820851688694, 'support': 4940.0} | {'precision': 0.8121475054229935, 'recall': 0.8555758683729433, 'f1-score': 0.8332962385933673, 'support': 2188.0} | {'precision': 0.9198786930150655, 'recall': 0.8978325217225246, 'f1-score': 0.9087219135056778, 'support': 10473.0} | {'precision': 0.8582063305978898, 'recall': 0.9208755267626895, 'f1-score': 0.8884371491853515, 'support': 15899.0} | 0.8524 | {'precision': 0.816162783421778, 'recall': 0.8021236107934867, 'f1-score': 0.8066593466133165, 'support': 33500.0} | {'precision': 0.8473766761482054, 'recall': 0.8523880597014926, 'f1-score': 0.8480805524125185, 'support': 33500.0} |
80
- | No log | 8.0 | 328 | 0.5458 | {'precision': 0.6705622932745314, 'recall': 0.6155870445344129, 'f1-score': 0.6418997361477573, 'support': 4940.0} | {'precision': 0.8129251700680272, 'recall': 0.8738574040219378, 'f1-score': 0.8422907488986784, 'support': 2188.0} | {'precision': 0.9259259259259259, 'recall': 0.89277188962093, 'f1-score': 0.909046716251033, 'support': 10473.0} | {'precision': 0.8728428701180745, 'recall': 0.9066607962764954, 'f1-score': 0.8894304929968533, 'support': 15899.0} | 0.8573 | {'precision': 0.8205640648466398, 'recall': 0.822219283613444, 'f1-score': 0.8206669235735804, 'support': 33500.0} | {'precision': 0.8556957914959558, 'recall': 0.8572537313432835, 'f1-score': 0.8559826424660975, 'support': 33500.0} |
81
- | No log | 9.0 | 369 | 0.5550 | {'precision': 0.6423661737138097, 'recall': 0.6242914979757085, 'f1-score': 0.6331998768093625, 'support': 4940.0} | {'precision': 0.8291592128801432, 'recall': 0.8473491773308958, 'f1-score': 0.8381555153707052, 'support': 2188.0} | {'precision': 0.909720885466795, 'recall': 0.9025112193258856, 'f1-score': 0.9061017111633034, 'support': 10473.0} | {'precision': 0.8796739874323399, 'recall': 0.8893012139128247, 'f1-score': 0.8844614037282621, 'support': 15899.0} | 0.8516 | {'precision': 0.8152300648732719, 'recall': 0.8158632771363286, 'f1-score': 0.8154796267679083, 'support': 33500.0} | {'precision': 0.8507741138987609, 'recall': 0.8516119402985075, 'f1-score': 0.8511506488942767, 'support': 33500.0} |
82
- | No log | 10.0 | 410 | 0.5788 | {'precision': 0.6611198560827524, 'recall': 0.5951417004048583, 'f1-score': 0.6263982102908278, 'support': 4940.0} | {'precision': 0.8315460232350312, 'recall': 0.8505484460694699, 'f1-score': 0.8409399005874378, 'support': 2188.0} | {'precision': 0.9248446592366111, 'recall': 0.8953499474840065, 'f1-score': 0.9098583349505143, 'support': 10473.0} | {'precision': 0.8645358599184456, 'recall': 0.9067865903515945, 'f1-score': 0.8851573292402148, 'support': 15899.0} | 0.8536 | {'precision': 0.8205115996182102, 'recall': 0.8119566710774824, 'f1-score': 0.8155884437672487, 'support': 33500.0} | {'precision': 0.851239060922849, 'recall': 0.8535820895522388, 'f1-score': 0.8518342203238483, 'support': 33500.0} |
83
- | No log | 11.0 | 451 | 0.5865 | {'precision': 0.661878453038674, 'recall': 0.6062753036437247, 'f1-score': 0.6328578975171685, 'support': 4940.0} | {'precision': 0.829535495179667, 'recall': 0.8651736745886655, 'f1-score': 0.8469798657718122, 'support': 2188.0} | {'precision': 0.9291244788564622, 'recall': 0.8937267258665139, 'f1-score': 0.9110819097678493, 'support': 10473.0} | {'precision': 0.8703893134364282, 'recall': 0.9098056481539719, 'f1-score': 0.88966111076942, 'support': 15899.0} | 0.8571 | {'precision': 0.8227319351278078, 'recall': 0.818745338063219, 'f1-score': 0.8201451959565625, 'support': 33500.0} | {'precision': 0.8553356293389153, 'recall': 0.8571044776119403, 'f1-score': 0.8557012776467233, 'support': 33500.0} |
84
- | No log | 12.0 | 492 | 0.6140 | {'precision': 0.6268885064065787, 'recall': 0.6635627530364372, 'f1-score': 0.6447044940505456, 'support': 4940.0} | {'precision': 0.8325078793336335, 'recall': 0.8450639853747715, 'f1-score': 0.8387389430709912, 'support': 2188.0} | {'precision': 0.923546196989078, 'recall': 0.896209300105032, 'f1-score': 0.9096724171351037, 'support': 10473.0} | {'precision': 0.885440926543715, 'recall': 0.8847726272092584, 'f1-score': 0.885106650726735, 'support': 15899.0} | 0.8531 | {'precision': 0.8170958773182513, 'recall': 0.8224021664313748, 'f1-score': 0.8195556262458439, 'support': 33500.0} | {'precision': 0.8557695842930038, 'recall': 0.8531343283582089, 'f1-score': 0.8543077872420695, 'support': 33500.0} |
85
- | 0.2701 | 13.0 | 533 | 0.6368 | {'precision': 0.6831773567678612, 'recall': 0.6058704453441296, 'f1-score': 0.642205771912885, 'support': 4940.0} | {'precision': 0.8174536256323778, 'recall': 0.8861974405850092, 'f1-score': 0.8504385964912281, 'support': 2188.0} | {'precision': 0.9274289099526066, 'recall': 0.8968776854769407, 'f1-score': 0.9118974807048201, 'support': 10473.0} | {'precision': 0.8733377459534268, 'recall': 0.912887602993899, 'f1-score': 0.892674826250077, 'support': 15899.0} | 0.8609 | {'precision': 0.8253494095765681, 'recall': 0.8254582935999946, 'f1-score': 0.8243041688397525, 'support': 33500.0} | {'precision': 0.8585565514078823, 'recall': 0.8608656716417911, 'f1-score': 0.8589909116520602, 'support': 33500.0} |
86
- | 0.2701 | 14.0 | 574 | 0.6486 | {'precision': 0.6641386782231853, 'recall': 0.6204453441295547, 'f1-score': 0.641548927263213, 'support': 4940.0} | {'precision': 0.8142076502732241, 'recall': 0.8852833638025595, 'f1-score': 0.8482592511495511, 'support': 2188.0} | {'precision': 0.9240070782540307, 'recall': 0.897450587224291, 'f1-score': 0.9105352385565513, 'support': 10473.0} | {'precision': 0.8767601322395004, 'recall': 0.9007484747468394, 'f1-score': 0.888592436323023, 'support': 15899.0} | 0.8574 | {'precision': 0.8197783847474851, 'recall': 0.8259819424758111, 'f1-score': 0.8222339633230846, 'support': 33500.0} | {'precision': 0.8560915487238994, 'recall': 0.8573731343283582, 'f1-score': 0.8563883474835222, 'support': 33500.0} |
87
- | 0.2701 | 15.0 | 615 | 0.6462 | {'precision': 0.6603214890016921, 'recall': 0.6319838056680162, 'f1-score': 0.6458419528340918, 'support': 4940.0} | {'precision': 0.8342832091188075, 'recall': 0.8697440585009141, 'f1-score': 0.8516446632356232, 'support': 2188.0} | {'precision': 0.9237646134197859, 'recall': 0.8978325217225246, 'f1-score': 0.9106139841177611, 'support': 10473.0} | {'precision': 0.8785556645414418, 'recall': 0.9013774451223348, 'f1-score': 0.8898202477414549, 'support': 15899.0} | 0.8585 | {'precision': 0.8242312440204318, 'recall': 0.8252344577534474, 'f1-score': 0.8244802119822328, 'support': 33500.0} | {'precision': 0.8576162126600033, 'recall': 0.8584776119402985, 'f1-score': 0.8578498550646764, 'support': 33500.0} |
88
- | 0.2701 | 16.0 | 656 | 0.6472 | {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0} | {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0} | {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0} | {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0} | 0.8588 | {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0} | {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[80%:100%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.8449255946993012
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6609
36
+ - Claim: {'precision': 0.6078710289236605, 'recall': 0.6151631477927063, 'f1-score': 0.6114953493918436, 'support': 4168.0}
37
+ - Majorclaim: {'precision': 0.782967032967033, 'recall': 0.7946096654275093, 'f1-score': 0.7887453874538746, 'support': 2152.0}
38
+ - O: {'precision': 0.934072084172823, 'recall': 0.9045089963147627, 'f1-score': 0.9190528634361235, 'support': 9226.0}
39
+ - Premise: {'precision': 0.8725067166001791, 'recall': 0.8876832601673155, 'f1-score': 0.8800295615043522, 'support': 12073.0}
40
+ - Accuracy: 0.8449
41
+ - Macro avg: {'precision': 0.7993542156659239, 'recall': 0.8004912674255735, 'f1-score': 0.7998307904465485, 'support': 27619.0}
42
+ - Weighted avg: {'precision': 0.8461593157460915, 'recall': 0.8449255946993012, 'f1-score': 0.8454278324403368, 'support': 27619.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.5690 | {'precision': 0.49395770392749244, 'recall': 0.23536468330134358, 'f1-score': 0.31881702957426067, 'support': 4168.0} | {'precision': 0.5330313325783315, 'recall': 0.6561338289962825, 'f1-score': 0.5882107894188711, 'support': 2152.0} | {'precision': 0.9200096957944491, 'recall': 0.82278343810969, 'f1-score': 0.8686845568461407, 'support': 9226.0} | {'precision': 0.777574153261386, 'recall': 0.9488942267870455, 'f1-score': 0.8547340147728121, 'support': 12073.0} | 0.7763 | {'precision': 0.6811432213904147, 'recall': 0.6657940442985903, 'f1-score': 0.6576115976530211, 'support': 27619.0} | {'precision': 0.7632992267425562, 'recall': 0.7762772004779318, 'f1-score': 0.7577517824653167, 'support': 27619.0} |
74
+ | No log | 2.0 | 82 | 0.4430 | {'precision': 0.6068347710683477, 'recall': 0.43881957773512476, 'f1-score': 0.5093288777499304, 'support': 4168.0} | {'precision': 0.6947840260798696, 'recall': 0.7922862453531598, 'f1-score': 0.7403386886669561, 'support': 2152.0} | {'precision': 0.930324074074074, 'recall': 0.8712334706264904, 'f1-score': 0.8998096943915818, 'support': 9226.0} | {'precision': 0.8270298275479239, 'recall': 0.9255363207156465, 'f1-score': 0.8735146966854284, 'support': 12073.0} | 0.8236 | {'precision': 0.7647431746925538, 'recall': 0.7569689036076054, 'f1-score': 0.7557479893734742, 'support': 27619.0} | {'precision': 0.8180007808150275, 'recall': 0.823563488902567, 'f1-score': 0.8169621924766614, 'support': 27619.0} |
75
+ | No log | 3.0 | 123 | 0.4280 | {'precision': 0.5555102040816327, 'recall': 0.6530710172744721, 'f1-score': 0.6003528892809882, 'support': 4168.0} | {'precision': 0.7618816682832201, 'recall': 0.7300185873605948, 'f1-score': 0.7456098718557191, 'support': 2152.0} | {'precision': 0.9472815190470575, 'recall': 0.8705831346195534, 'f1-score': 0.9073143179892686, 'support': 9226.0} | {'precision': 0.8730497618656594, 'recall': 0.8806427565642343, 'f1-score': 0.8768298214506619, 'support': 12073.0} | 0.8312 | {'precision': 0.7844307883193924, 'recall': 0.7835788739547136, 'f1-score': 0.7825267251441594, 'support': 27619.0} | {'precision': 0.8412645262496828, 'recall': 0.8312031572468228, 'f1-score': 0.8350654121763821, 'support': 27619.0} |
76
+ | No log | 4.0 | 164 | 0.4198 | {'precision': 0.6521200866604766, 'recall': 0.5055182341650671, 'f1-score': 0.5695364238410595, 'support': 4168.0} | {'precision': 0.7789709172259508, 'recall': 0.8090148698884758, 'f1-score': 0.7937086847503988, 'support': 2152.0} | {'precision': 0.91675722668985, 'recall': 0.9143724257533059, 'f1-score': 0.9155632732797916, 'support': 9226.0} | {'precision': 0.85398810902633, 'recall': 0.9160937629421022, 'f1-score': 0.8839514066496164, 'support': 12073.0} | 0.8452 | {'precision': 0.8004590849006519, 'recall': 0.7862498231872379, 'f1-score': 0.7906899471302166, 'support': 27619.0} | {'precision': 0.8386466761572305, 'recall': 0.8452152503711213, 'f1-score': 0.8400311740436862, 'support': 27619.0} |
77
+ | No log | 5.0 | 205 | 0.4471 | {'precision': 0.5814893617021276, 'recall': 0.6557101727447217, 'f1-score': 0.6163734776725303, 'support': 4168.0} | {'precision': 0.7235804416403786, 'recall': 0.8526951672862454, 'f1-score': 0.7828498293515358, 'support': 2152.0} | {'precision': 0.9300457436126297, 'recall': 0.9035334923043572, 'f1-score': 0.9165979438121942, 'support': 9226.0} | {'precision': 0.9016637478108581, 'recall': 0.8528948894226787, 'f1-score': 0.8766015408845188, 'support': 12073.0} | 0.8400 | {'precision': 0.7841948236914985, 'recall': 0.8162084304395008, 'f1-score': 0.7981056979301948, 'support': 27619.0} | {'precision': 0.8489511288560475, 'recall': 0.8400376552373366, 'f1-score': 0.843386093646175, 'support': 27619.0} |
78
+ | No log | 6.0 | 246 | 0.4595 | {'precision': 0.5807517554729451, 'recall': 0.6746641074856046, 'f1-score': 0.6241953385127637, 'support': 4168.0} | {'precision': 0.7883110906580764, 'recall': 0.796003717472119, 'f1-score': 0.7921387283236995, 'support': 2152.0} | {'precision': 0.9110802732707088, 'recall': 0.925102969867765, 'f1-score': 0.9180380767989674, 'support': 9226.0} | {'precision': 0.9042363830544677, 'recall': 0.8415472542035948, 'f1-score': 0.8717662705392766, 'support': 12073.0} | 0.8407 | {'precision': 0.7960948756140495, 'recall': 0.8093295122572709, 'f1-score': 0.8015346035436768, 'support': 27619.0} | {'precision': 0.8486726976979458, 'recall': 0.8407255874579094, 'f1-score': 0.8436577064716956, 'support': 27619.0} |
79
+ | No log | 7.0 | 287 | 0.5069 | {'precision': 0.6110236220472441, 'recall': 0.5585412667946257, 'f1-score': 0.5836049135121585, 'support': 4168.0} | {'precision': 0.8053691275167785, 'recall': 0.7806691449814126, 'f1-score': 0.7928268050967437, 'support': 2152.0} | {'precision': 0.9251618566882476, 'recall': 0.9138304790808585, 'f1-score': 0.9194612574295218, 'support': 9226.0} | {'precision': 0.8609833465503569, 'recall': 0.8992793837488611, 'f1-score': 0.8797147834541992, 'support': 12073.0} | 0.8435 | {'precision': 0.8006344882006567, 'recall': 0.7880800686514394, 'f1-score': 0.7939019398731558, 'support': 27619.0} | {'precision': 0.8403669956123412, 'recall': 0.8434773163402006, 'f1-score': 0.8415357075120093, 'support': 27619.0} |
80
+ | No log | 8.0 | 328 | 0.5486 | {'precision': 0.5794648982391951, 'recall': 0.6079654510556622, 'f1-score': 0.5933731413183467, 'support': 4168.0} | {'precision': 0.7641959254442999, 'recall': 0.8192379182156134, 'f1-score': 0.7907602601480151, 'support': 2152.0} | {'precision': 0.9482497964879637, 'recall': 0.8838066334272707, 'f1-score': 0.9148948106591865, 'support': 9226.0} | {'precision': 0.86709886547812, 'recall': 0.8862751594466992, 'f1-score': 0.8765821488551183, 'support': 12073.0} | 0.8382 | {'precision': 0.7897523714123946, 'recall': 0.7993212905363115, 'f1-score': 0.7939025902451666, 'support': 27619.0} | {'precision': 0.8427820179127555, 'recall': 0.8382273072884608, 'f1-score': 0.8399540584062747, 'support': 27619.0} |
81
+ | No log | 9.0 | 369 | 0.5624 | {'precision': 0.5684468999386126, 'recall': 0.6665067178502879, 'f1-score': 0.6135836554389841, 'support': 4168.0} | {'precision': 0.7784669915817457, 'recall': 0.8164498141263941, 'f1-score': 0.7970061238376048, 'support': 2152.0} | {'precision': 0.9420438957475995, 'recall': 0.893236505527856, 'f1-score': 0.9169912095248693, 'support': 9226.0} | {'precision': 0.8849663170461328, 'recall': 0.8596040752091444, 'f1-score': 0.8721008403361344, 'support': 12073.0} | 0.8383 | {'precision': 0.7934810260785227, 'recall': 0.8089492781784205, 'f1-score': 0.7999204572843982, 'support': 27619.0} | {'precision': 0.8479685351639584, 'recall': 0.8383359281653934, 'f1-score': 0.8422320938058151, 'support': 27619.0} |
82
+ | No log | 10.0 | 410 | 0.5923 | {'precision': 0.6067892503536068, 'recall': 0.6175623800383877, 'f1-score': 0.612128418549346, 'support': 4168.0} | {'precision': 0.7623089983022071, 'recall': 0.8345724907063197, 'f1-score': 0.7968056787932565, 'support': 2152.0} | {'precision': 0.9368265850062379, 'recall': 0.8952959028831563, 'f1-score': 0.9155905337249902, 'support': 9226.0} | {'precision': 0.8744673877417241, 'recall': 0.8839559347303901, 'f1-score': 0.879186060880669, 'support': 12073.0} | 0.8437 | {'precision': 0.795098055350944, 'recall': 0.8078466770895635, 'f1-score': 0.8009276729870654, 'support': 27619.0} | {'precision': 0.846163633922067, 'recall': 0.8436945580940657, 'f1-score': 0.8446261141401151, 'support': 27619.0} |
83
+ | No log | 11.0 | 451 | 0.6036 | {'precision': 0.5938604240282686, 'recall': 0.6451535508637236, 'f1-score': 0.6184452621895125, 'support': 4168.0} | {'precision': 0.7668161434977578, 'recall': 0.7946096654275093, 'f1-score': 0.7804655408489276, 'support': 2152.0} | {'precision': 0.9390562819783969, 'recall': 0.8951875135486668, 'f1-score': 0.9165973031463293, 'support': 9226.0} | {'precision': 0.8781700646444555, 'recall': 0.8776608962146939, 'f1-score': 0.8779154066034218, 'support': 12073.0} | 0.8420 | {'precision': 0.7944757285372197, 'recall': 0.8031529065136485, 'f1-score': 0.7983558781970478, 'support': 27619.0} | {'precision': 0.8469270804932185, 'recall': 0.841956624063145, 'f1-score': 0.8440870820617664, 'support': 27619.0} |
84
+ | No log | 12.0 | 492 | 0.6292 | {'precision': 0.594930767425487, 'recall': 0.6082053742802304, 'f1-score': 0.6014948392454621, 'support': 4168.0} | {'precision': 0.7890961262553802, 'recall': 0.766728624535316, 'f1-score': 0.7777515908555267, 'support': 2152.0} | {'precision': 0.9292805354155047, 'recall': 0.9029915456319099, 'f1-score': 0.9159474465394976, 'support': 9226.0} | {'precision': 0.872541050235734, 'recall': 0.8890913608879317, 'f1-score': 0.8807384615384615, 'support': 12073.0} | 0.8418 | {'precision': 0.7964621198330264, 'recall': 0.791754226333847, 'f1-score': 0.7939830845447369, 'support': 27619.0} | {'precision': 0.8430984692266364, 'recall': 0.8418117962272349, 'f1-score': 0.8423345704559697, 'support': 27619.0} |
85
+ | 0.2689 | 13.0 | 533 | 0.6506 | {'precision': 0.6016401590457257, 'recall': 0.5808541266794626, 'f1-score': 0.5910644531250001, 'support': 4168.0} | {'precision': 0.7968977217644208, 'recall': 0.7639405204460966, 'f1-score': 0.7800711743772243, 'support': 2152.0} | {'precision': 0.9178990865593737, 'recall': 0.9149143724257534, 'f1-score': 0.9164042992074695, 'support': 9226.0} | {'precision': 0.8670557717250325, 'recall': 0.8859438416300837, 'f1-score': 0.8763980498996273, 'support': 12073.0} | 0.8401 | {'precision': 0.7958731847736381, 'recall': 0.786413215295349, 'f1-score': 0.7909844941523303, 'support': 27619.0} | {'precision': 0.8385191855162285, 'recall': 0.8400738621963141, 'f1-score': 0.8391965505199718, 'support': 27619.0} |
86
+ | 0.2689 | 14.0 | 574 | 0.6476 | {'precision': 0.6124620060790273, 'recall': 0.5801343570057581, 'f1-score': 0.5958600295712174, 'support': 4168.0} | {'precision': 0.77728285077951, 'recall': 0.8108736059479554, 'f1-score': 0.7937229929497385, 'support': 2152.0} | {'precision': 0.9248128577719067, 'recall': 0.9105787990461739, 'f1-score': 0.9176406335335883, 'support': 9226.0} | {'precision': 0.8699562469615946, 'recall': 0.8893398492503934, 'f1-score': 0.8795412656154004, 'support': 12073.0} | 0.8437 | {'precision': 0.7961284903980096, 'recall': 0.7977316528125702, 'f1-score': 0.7966912304174861, 'support': 27619.0} | {'precision': 0.8422013661459805, 'recall': 0.8436583511350881, 'f1-score': 0.8427709427870771, 'support': 27619.0} |
87
+ | 0.2689 | 15.0 | 615 | 0.6652 | {'precision': 0.5992348158775705, 'recall': 0.6012476007677543, 'f1-score': 0.6002395209580837, 'support': 4168.0} | {'precision': 0.7798372513562387, 'recall': 0.8015799256505576, 'f1-score': 0.7905591200733272, 'support': 2152.0} | {'precision': 0.9388219240391176, 'recall': 0.8948623455451984, 'f1-score': 0.916315205327414, 'support': 9226.0} | {'precision': 0.8667846512750382, 'recall': 0.8924873685082415, 'f1-score': 0.8794482533463925, 'support': 12073.0} | 0.8422 | {'precision': 0.7961696606369912, 'recall': 0.797544310117938, 'f1-score': 0.7966405249263044, 'support': 27619.0} | {'precision': 0.8436975503647769, 'recall': 0.842246279734965, 'f1-score': 0.842701922471951, 'support': 27619.0} |
88
+ | 0.2689 | 16.0 | 656 | 0.6609 | {'precision': 0.6078710289236605, 'recall': 0.6151631477927063, 'f1-score': 0.6114953493918436, 'support': 4168.0} | {'precision': 0.782967032967033, 'recall': 0.7946096654275093, 'f1-score': 0.7887453874538746, 'support': 2152.0} | {'precision': 0.934072084172823, 'recall': 0.9045089963147627, 'f1-score': 0.9190528634361235, 'support': 9226.0} | {'precision': 0.8725067166001791, 'recall': 0.8876832601673155, 'f1-score': 0.8800295615043522, 'support': 12073.0} | 0.8449 | {'precision': 0.7993542156659239, 'recall': 0.8004912674255735, 'f1-score': 0.7998307904465485, 'support': 27619.0} | {'precision': 0.8461593157460915, 'recall': 0.8449255946993012, 'f1-score': 0.8454278324403368, 'support': 27619.0} |
89
 
90
 
91
  ### Framework versions
meta_data/README_s42_e16.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[60%:80%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.858776119402985
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6472
36
- - Claim: {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0}
37
- - Majorclaim: {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0}
38
- - O: {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0}
39
- - Premise: {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0}
40
- - Accuracy: 0.8588
41
- - Macro avg: {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0}
42
- - Weighted avg: {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6237 | {'precision': 0.4813399941228328, 'recall': 0.33157894736842103, 'f1-score': 0.3926645091693635, 'support': 4940.0} | {'precision': 0.41758530183727033, 'recall': 0.7271480804387569, 'f1-score': 0.5305101700566855, 'support': 2188.0} | {'precision': 0.8614998552263295, 'recall': 0.8522868328081734, 'f1-score': 0.8568685802054334, 'support': 10473.0} | {'precision': 0.8528192892126083, 'recall': 0.8542675639977357, 'f1-score': 0.8535428122545169, 'support': 15899.0} | 0.7683 | {'precision': 0.6533111100997602, 'recall': 0.6913203561532717, 'f1-score': 0.6583965179214999, 'support': 33500.0} | {'precision': 0.7723271066974134, 'recall': 0.7682686567164179, 'f1-score': 0.7655218131315448, 'support': 33500.0} |
74
- | No log | 2.0 | 82 | 0.4751 | {'precision': 0.5846230654018971, 'recall': 0.47408906882591095, 'f1-score': 0.5235859602056785, 'support': 4940.0} | {'precision': 0.7269767441860465, 'recall': 0.7143510054844607, 'f1-score': 0.7206085753803596, 'support': 2188.0} | {'precision': 0.9142337609859582, 'recall': 0.8641268022534135, 'f1-score': 0.8884743765953269, 'support': 10473.0} | {'precision': 0.8357695614789338, 'recall': 0.917038807472168, 'f1-score': 0.8745201535508637, 'support': 15899.0} | 0.8219 | {'precision': 0.7654007830132088, 'recall': 0.7424014210089883, 'f1-score': 0.7517972664330571, 'support': 33500.0} | {'precision': 0.816159208839521, 'recall': 0.8219402985074626, 'f1-score': 0.8170804260816812, 'support': 33500.0} |
75
- | No log | 3.0 | 123 | 0.4586 | {'precision': 0.6658894070619586, 'recall': 0.4046558704453441, 'f1-score': 0.5033996474439688, 'support': 4940.0} | {'precision': 0.7872244714349977, 'recall': 0.79981718464351, 'f1-score': 0.7934708682838358, 'support': 2188.0} | {'precision': 0.9342819121711536, 'recall': 0.8714790413444095, 'f1-score': 0.9017883608339096, 'support': 10473.0} | {'precision': 0.8168702042580784, 'recall': 0.9508145166362665, 'f1-score': 0.8787676209853219, 'support': 15899.0} | 0.8356 | {'precision': 0.8010664987315472, 'recall': 0.7566916532673825, 'f1-score': 0.7693566243867591, 'support': 33500.0} | {'precision': 0.8293759599418965, 'recall': 0.8356119402985075, 'f1-score': 0.8250407291712659, 'support': 33500.0} |
76
- | No log | 4.0 | 164 | 0.4525 | {'precision': 0.5575898801597869, 'recall': 0.6781376518218624, 'f1-score': 0.6119839240043845, 'support': 4940.0} | {'precision': 0.7466456195737964, 'recall': 0.8647166361974405, 'f1-score': 0.8013553578991952, 'support': 2188.0} | {'precision': 0.9201592832254853, 'recall': 0.8825551417931825, 'f1-score': 0.9009650063359004, 'support': 10473.0} | {'precision': 0.8922416683430564, 'recall': 0.836907981634065, 'f1-score': 0.8636894716344281, 'support': 15899.0} | 0.8296 | {'precision': 0.7791591128255312, 'recall': 0.8155793528616375, 'f1-score': 0.7944984399684771, 'support': 33500.0} | {'precision': 0.8421114352783158, 'recall': 0.8295820895522388, 'f1-score': 0.8341543739861718, 'support': 33500.0} |
77
- | No log | 5.0 | 205 | 0.4721 | {'precision': 0.662877030162413, 'recall': 0.5783400809716599, 'f1-score': 0.6177297297297297, 'support': 4940.0} | {'precision': 0.7945205479452054, 'recall': 0.8747714808043876, 'f1-score': 0.8327169893408746, 'support': 2188.0} | {'precision': 0.9125229313507772, 'recall': 0.9024157357013273, 'f1-score': 0.9074411905904946, 'support': 10473.0} | {'precision': 0.8726254262055528, 'recall': 0.9014403421598842, 'f1-score': 0.8867988738669058, 'support': 15899.0} | 0.8524 | {'precision': 0.8106364839159872, 'recall': 0.8142419099093148, 'f1-score': 0.8111716958820011, 'support': 33500.0} | {'precision': 0.8490670984831405, 'recall': 0.8523582089552239, 'f1-score': 0.8500422842449816, 'support': 33500.0} |
78
- | No log | 6.0 | 246 | 0.4792 | {'precision': 0.6428419936373276, 'recall': 0.6135627530364373, 'f1-score': 0.6278612118073537, 'support': 4940.0} | {'precision': 0.804950917626974, 'recall': 0.8619744058500914, 'f1-score': 0.83248730964467, 'support': 2188.0} | {'precision': 0.9285714285714286, 'recall': 0.8949680129857729, 'f1-score': 0.9114601059950406, 'support': 10473.0} | {'precision': 0.872155615365794, 'recall': 0.8967859613812189, 'f1-score': 0.8842993146649301, 'support': 15899.0} | 0.8522 | {'precision': 0.812129988800381, 'recall': 0.8168227833133802, 'f1-score': 0.8140269855279986, 'support': 33500.0} | {'precision': 0.8515881419840463, 'recall': 0.8521791044776119, 'f1-score': 0.8515914362320791, 'support': 33500.0} |
79
- | No log | 7.0 | 287 | 0.5202 | {'precision': 0.6744186046511628, 'recall': 0.5342105263157895, 'f1-score': 0.5961820851688694, 'support': 4940.0} | {'precision': 0.8121475054229935, 'recall': 0.8555758683729433, 'f1-score': 0.8332962385933673, 'support': 2188.0} | {'precision': 0.9198786930150655, 'recall': 0.8978325217225246, 'f1-score': 0.9087219135056778, 'support': 10473.0} | {'precision': 0.8582063305978898, 'recall': 0.9208755267626895, 'f1-score': 0.8884371491853515, 'support': 15899.0} | 0.8524 | {'precision': 0.816162783421778, 'recall': 0.8021236107934867, 'f1-score': 0.8066593466133165, 'support': 33500.0} | {'precision': 0.8473766761482054, 'recall': 0.8523880597014926, 'f1-score': 0.8480805524125185, 'support': 33500.0} |
80
- | No log | 8.0 | 328 | 0.5458 | {'precision': 0.6705622932745314, 'recall': 0.6155870445344129, 'f1-score': 0.6418997361477573, 'support': 4940.0} | {'precision': 0.8129251700680272, 'recall': 0.8738574040219378, 'f1-score': 0.8422907488986784, 'support': 2188.0} | {'precision': 0.9259259259259259, 'recall': 0.89277188962093, 'f1-score': 0.909046716251033, 'support': 10473.0} | {'precision': 0.8728428701180745, 'recall': 0.9066607962764954, 'f1-score': 0.8894304929968533, 'support': 15899.0} | 0.8573 | {'precision': 0.8205640648466398, 'recall': 0.822219283613444, 'f1-score': 0.8206669235735804, 'support': 33500.0} | {'precision': 0.8556957914959558, 'recall': 0.8572537313432835, 'f1-score': 0.8559826424660975, 'support': 33500.0} |
81
- | No log | 9.0 | 369 | 0.5550 | {'precision': 0.6423661737138097, 'recall': 0.6242914979757085, 'f1-score': 0.6331998768093625, 'support': 4940.0} | {'precision': 0.8291592128801432, 'recall': 0.8473491773308958, 'f1-score': 0.8381555153707052, 'support': 2188.0} | {'precision': 0.909720885466795, 'recall': 0.9025112193258856, 'f1-score': 0.9061017111633034, 'support': 10473.0} | {'precision': 0.8796739874323399, 'recall': 0.8893012139128247, 'f1-score': 0.8844614037282621, 'support': 15899.0} | 0.8516 | {'precision': 0.8152300648732719, 'recall': 0.8158632771363286, 'f1-score': 0.8154796267679083, 'support': 33500.0} | {'precision': 0.8507741138987609, 'recall': 0.8516119402985075, 'f1-score': 0.8511506488942767, 'support': 33500.0} |
82
- | No log | 10.0 | 410 | 0.5788 | {'precision': 0.6611198560827524, 'recall': 0.5951417004048583, 'f1-score': 0.6263982102908278, 'support': 4940.0} | {'precision': 0.8315460232350312, 'recall': 0.8505484460694699, 'f1-score': 0.8409399005874378, 'support': 2188.0} | {'precision': 0.9248446592366111, 'recall': 0.8953499474840065, 'f1-score': 0.9098583349505143, 'support': 10473.0} | {'precision': 0.8645358599184456, 'recall': 0.9067865903515945, 'f1-score': 0.8851573292402148, 'support': 15899.0} | 0.8536 | {'precision': 0.8205115996182102, 'recall': 0.8119566710774824, 'f1-score': 0.8155884437672487, 'support': 33500.0} | {'precision': 0.851239060922849, 'recall': 0.8535820895522388, 'f1-score': 0.8518342203238483, 'support': 33500.0} |
83
- | No log | 11.0 | 451 | 0.5865 | {'precision': 0.661878453038674, 'recall': 0.6062753036437247, 'f1-score': 0.6328578975171685, 'support': 4940.0} | {'precision': 0.829535495179667, 'recall': 0.8651736745886655, 'f1-score': 0.8469798657718122, 'support': 2188.0} | {'precision': 0.9291244788564622, 'recall': 0.8937267258665139, 'f1-score': 0.9110819097678493, 'support': 10473.0} | {'precision': 0.8703893134364282, 'recall': 0.9098056481539719, 'f1-score': 0.88966111076942, 'support': 15899.0} | 0.8571 | {'precision': 0.8227319351278078, 'recall': 0.818745338063219, 'f1-score': 0.8201451959565625, 'support': 33500.0} | {'precision': 0.8553356293389153, 'recall': 0.8571044776119403, 'f1-score': 0.8557012776467233, 'support': 33500.0} |
84
- | No log | 12.0 | 492 | 0.6140 | {'precision': 0.6268885064065787, 'recall': 0.6635627530364372, 'f1-score': 0.6447044940505456, 'support': 4940.0} | {'precision': 0.8325078793336335, 'recall': 0.8450639853747715, 'f1-score': 0.8387389430709912, 'support': 2188.0} | {'precision': 0.923546196989078, 'recall': 0.896209300105032, 'f1-score': 0.9096724171351037, 'support': 10473.0} | {'precision': 0.885440926543715, 'recall': 0.8847726272092584, 'f1-score': 0.885106650726735, 'support': 15899.0} | 0.8531 | {'precision': 0.8170958773182513, 'recall': 0.8224021664313748, 'f1-score': 0.8195556262458439, 'support': 33500.0} | {'precision': 0.8557695842930038, 'recall': 0.8531343283582089, 'f1-score': 0.8543077872420695, 'support': 33500.0} |
85
- | 0.2701 | 13.0 | 533 | 0.6368 | {'precision': 0.6831773567678612, 'recall': 0.6058704453441296, 'f1-score': 0.642205771912885, 'support': 4940.0} | {'precision': 0.8174536256323778, 'recall': 0.8861974405850092, 'f1-score': 0.8504385964912281, 'support': 2188.0} | {'precision': 0.9274289099526066, 'recall': 0.8968776854769407, 'f1-score': 0.9118974807048201, 'support': 10473.0} | {'precision': 0.8733377459534268, 'recall': 0.912887602993899, 'f1-score': 0.892674826250077, 'support': 15899.0} | 0.8609 | {'precision': 0.8253494095765681, 'recall': 0.8254582935999946, 'f1-score': 0.8243041688397525, 'support': 33500.0} | {'precision': 0.8585565514078823, 'recall': 0.8608656716417911, 'f1-score': 0.8589909116520602, 'support': 33500.0} |
86
- | 0.2701 | 14.0 | 574 | 0.6486 | {'precision': 0.6641386782231853, 'recall': 0.6204453441295547, 'f1-score': 0.641548927263213, 'support': 4940.0} | {'precision': 0.8142076502732241, 'recall': 0.8852833638025595, 'f1-score': 0.8482592511495511, 'support': 2188.0} | {'precision': 0.9240070782540307, 'recall': 0.897450587224291, 'f1-score': 0.9105352385565513, 'support': 10473.0} | {'precision': 0.8767601322395004, 'recall': 0.9007484747468394, 'f1-score': 0.888592436323023, 'support': 15899.0} | 0.8574 | {'precision': 0.8197783847474851, 'recall': 0.8259819424758111, 'f1-score': 0.8222339633230846, 'support': 33500.0} | {'precision': 0.8560915487238994, 'recall': 0.8573731343283582, 'f1-score': 0.8563883474835222, 'support': 33500.0} |
87
- | 0.2701 | 15.0 | 615 | 0.6462 | {'precision': 0.6603214890016921, 'recall': 0.6319838056680162, 'f1-score': 0.6458419528340918, 'support': 4940.0} | {'precision': 0.8342832091188075, 'recall': 0.8697440585009141, 'f1-score': 0.8516446632356232, 'support': 2188.0} | {'precision': 0.9237646134197859, 'recall': 0.8978325217225246, 'f1-score': 0.9106139841177611, 'support': 10473.0} | {'precision': 0.8785556645414418, 'recall': 0.9013774451223348, 'f1-score': 0.8898202477414549, 'support': 15899.0} | 0.8585 | {'precision': 0.8242312440204318, 'recall': 0.8252344577534474, 'f1-score': 0.8244802119822328, 'support': 33500.0} | {'precision': 0.8576162126600033, 'recall': 0.8584776119402985, 'f1-score': 0.8578498550646764, 'support': 33500.0} |
88
- | 0.2701 | 16.0 | 656 | 0.6472 | {'precision': 0.6572622779519331, 'recall': 0.6366396761133604, 'f1-score': 0.6467866323907456, 'support': 4940.0} | {'precision': 0.8274678111587983, 'recall': 0.8811700182815356, 'f1-score': 0.8534749889331562, 'support': 2188.0} | {'precision': 0.9268028016178357, 'recall': 0.8970686527260575, 'f1-score': 0.9116933527413877, 'support': 10473.0} | {'precision': 0.8801698670605613, 'recall': 0.8994905339958488, 'f1-score': 0.8897253242915357, 'support': 15899.0} | 0.8588 | {'precision': 0.8229256894472821, 'recall': 0.8285922202792007, 'f1-score': 0.8254200745892063, 'support': 33500.0} | {'precision': 0.8584358710936555, 'recall': 0.858776119402985, 'f1-score': 0.8584010941482899, 'support': 33500.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[80%:100%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.8449255946993012
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6609
36
+ - Claim: {'precision': 0.6078710289236605, 'recall': 0.6151631477927063, 'f1-score': 0.6114953493918436, 'support': 4168.0}
37
+ - Majorclaim: {'precision': 0.782967032967033, 'recall': 0.7946096654275093, 'f1-score': 0.7887453874538746, 'support': 2152.0}
38
+ - O: {'precision': 0.934072084172823, 'recall': 0.9045089963147627, 'f1-score': 0.9190528634361235, 'support': 9226.0}
39
+ - Premise: {'precision': 0.8725067166001791, 'recall': 0.8876832601673155, 'f1-score': 0.8800295615043522, 'support': 12073.0}
40
+ - Accuracy: 0.8449
41
+ - Macro avg: {'precision': 0.7993542156659239, 'recall': 0.8004912674255735, 'f1-score': 0.7998307904465485, 'support': 27619.0}
42
+ - Weighted avg: {'precision': 0.8461593157460915, 'recall': 0.8449255946993012, 'f1-score': 0.8454278324403368, 'support': 27619.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.5690 | {'precision': 0.49395770392749244, 'recall': 0.23536468330134358, 'f1-score': 0.31881702957426067, 'support': 4168.0} | {'precision': 0.5330313325783315, 'recall': 0.6561338289962825, 'f1-score': 0.5882107894188711, 'support': 2152.0} | {'precision': 0.9200096957944491, 'recall': 0.82278343810969, 'f1-score': 0.8686845568461407, 'support': 9226.0} | {'precision': 0.777574153261386, 'recall': 0.9488942267870455, 'f1-score': 0.8547340147728121, 'support': 12073.0} | 0.7763 | {'precision': 0.6811432213904147, 'recall': 0.6657940442985903, 'f1-score': 0.6576115976530211, 'support': 27619.0} | {'precision': 0.7632992267425562, 'recall': 0.7762772004779318, 'f1-score': 0.7577517824653167, 'support': 27619.0} |
74
+ | No log | 2.0 | 82 | 0.4430 | {'precision': 0.6068347710683477, 'recall': 0.43881957773512476, 'f1-score': 0.5093288777499304, 'support': 4168.0} | {'precision': 0.6947840260798696, 'recall': 0.7922862453531598, 'f1-score': 0.7403386886669561, 'support': 2152.0} | {'precision': 0.930324074074074, 'recall': 0.8712334706264904, 'f1-score': 0.8998096943915818, 'support': 9226.0} | {'precision': 0.8270298275479239, 'recall': 0.9255363207156465, 'f1-score': 0.8735146966854284, 'support': 12073.0} | 0.8236 | {'precision': 0.7647431746925538, 'recall': 0.7569689036076054, 'f1-score': 0.7557479893734742, 'support': 27619.0} | {'precision': 0.8180007808150275, 'recall': 0.823563488902567, 'f1-score': 0.8169621924766614, 'support': 27619.0} |
75
+ | No log | 3.0 | 123 | 0.4280 | {'precision': 0.5555102040816327, 'recall': 0.6530710172744721, 'f1-score': 0.6003528892809882, 'support': 4168.0} | {'precision': 0.7618816682832201, 'recall': 0.7300185873605948, 'f1-score': 0.7456098718557191, 'support': 2152.0} | {'precision': 0.9472815190470575, 'recall': 0.8705831346195534, 'f1-score': 0.9073143179892686, 'support': 9226.0} | {'precision': 0.8730497618656594, 'recall': 0.8806427565642343, 'f1-score': 0.8768298214506619, 'support': 12073.0} | 0.8312 | {'precision': 0.7844307883193924, 'recall': 0.7835788739547136, 'f1-score': 0.7825267251441594, 'support': 27619.0} | {'precision': 0.8412645262496828, 'recall': 0.8312031572468228, 'f1-score': 0.8350654121763821, 'support': 27619.0} |
76
+ | No log | 4.0 | 164 | 0.4198 | {'precision': 0.6521200866604766, 'recall': 0.5055182341650671, 'f1-score': 0.5695364238410595, 'support': 4168.0} | {'precision': 0.7789709172259508, 'recall': 0.8090148698884758, 'f1-score': 0.7937086847503988, 'support': 2152.0} | {'precision': 0.91675722668985, 'recall': 0.9143724257533059, 'f1-score': 0.9155632732797916, 'support': 9226.0} | {'precision': 0.85398810902633, 'recall': 0.9160937629421022, 'f1-score': 0.8839514066496164, 'support': 12073.0} | 0.8452 | {'precision': 0.8004590849006519, 'recall': 0.7862498231872379, 'f1-score': 0.7906899471302166, 'support': 27619.0} | {'precision': 0.8386466761572305, 'recall': 0.8452152503711213, 'f1-score': 0.8400311740436862, 'support': 27619.0} |
77
+ | No log | 5.0 | 205 | 0.4471 | {'precision': 0.5814893617021276, 'recall': 0.6557101727447217, 'f1-score': 0.6163734776725303, 'support': 4168.0} | {'precision': 0.7235804416403786, 'recall': 0.8526951672862454, 'f1-score': 0.7828498293515358, 'support': 2152.0} | {'precision': 0.9300457436126297, 'recall': 0.9035334923043572, 'f1-score': 0.9165979438121942, 'support': 9226.0} | {'precision': 0.9016637478108581, 'recall': 0.8528948894226787, 'f1-score': 0.8766015408845188, 'support': 12073.0} | 0.8400 | {'precision': 0.7841948236914985, 'recall': 0.8162084304395008, 'f1-score': 0.7981056979301948, 'support': 27619.0} | {'precision': 0.8489511288560475, 'recall': 0.8400376552373366, 'f1-score': 0.843386093646175, 'support': 27619.0} |
78
+ | No log | 6.0 | 246 | 0.4595 | {'precision': 0.5807517554729451, 'recall': 0.6746641074856046, 'f1-score': 0.6241953385127637, 'support': 4168.0} | {'precision': 0.7883110906580764, 'recall': 0.796003717472119, 'f1-score': 0.7921387283236995, 'support': 2152.0} | {'precision': 0.9110802732707088, 'recall': 0.925102969867765, 'f1-score': 0.9180380767989674, 'support': 9226.0} | {'precision': 0.9042363830544677, 'recall': 0.8415472542035948, 'f1-score': 0.8717662705392766, 'support': 12073.0} | 0.8407 | {'precision': 0.7960948756140495, 'recall': 0.8093295122572709, 'f1-score': 0.8015346035436768, 'support': 27619.0} | {'precision': 0.8486726976979458, 'recall': 0.8407255874579094, 'f1-score': 0.8436577064716956, 'support': 27619.0} |
79
+ | No log | 7.0 | 287 | 0.5069 | {'precision': 0.6110236220472441, 'recall': 0.5585412667946257, 'f1-score': 0.5836049135121585, 'support': 4168.0} | {'precision': 0.8053691275167785, 'recall': 0.7806691449814126, 'f1-score': 0.7928268050967437, 'support': 2152.0} | {'precision': 0.9251618566882476, 'recall': 0.9138304790808585, 'f1-score': 0.9194612574295218, 'support': 9226.0} | {'precision': 0.8609833465503569, 'recall': 0.8992793837488611, 'f1-score': 0.8797147834541992, 'support': 12073.0} | 0.8435 | {'precision': 0.8006344882006567, 'recall': 0.7880800686514394, 'f1-score': 0.7939019398731558, 'support': 27619.0} | {'precision': 0.8403669956123412, 'recall': 0.8434773163402006, 'f1-score': 0.8415357075120093, 'support': 27619.0} |
80
+ | No log | 8.0 | 328 | 0.5486 | {'precision': 0.5794648982391951, 'recall': 0.6079654510556622, 'f1-score': 0.5933731413183467, 'support': 4168.0} | {'precision': 0.7641959254442999, 'recall': 0.8192379182156134, 'f1-score': 0.7907602601480151, 'support': 2152.0} | {'precision': 0.9482497964879637, 'recall': 0.8838066334272707, 'f1-score': 0.9148948106591865, 'support': 9226.0} | {'precision': 0.86709886547812, 'recall': 0.8862751594466992, 'f1-score': 0.8765821488551183, 'support': 12073.0} | 0.8382 | {'precision': 0.7897523714123946, 'recall': 0.7993212905363115, 'f1-score': 0.7939025902451666, 'support': 27619.0} | {'precision': 0.8427820179127555, 'recall': 0.8382273072884608, 'f1-score': 0.8399540584062747, 'support': 27619.0} |
81
+ | No log | 9.0 | 369 | 0.5624 | {'precision': 0.5684468999386126, 'recall': 0.6665067178502879, 'f1-score': 0.6135836554389841, 'support': 4168.0} | {'precision': 0.7784669915817457, 'recall': 0.8164498141263941, 'f1-score': 0.7970061238376048, 'support': 2152.0} | {'precision': 0.9420438957475995, 'recall': 0.893236505527856, 'f1-score': 0.9169912095248693, 'support': 9226.0} | {'precision': 0.8849663170461328, 'recall': 0.8596040752091444, 'f1-score': 0.8721008403361344, 'support': 12073.0} | 0.8383 | {'precision': 0.7934810260785227, 'recall': 0.8089492781784205, 'f1-score': 0.7999204572843982, 'support': 27619.0} | {'precision': 0.8479685351639584, 'recall': 0.8383359281653934, 'f1-score': 0.8422320938058151, 'support': 27619.0} |
82
+ | No log | 10.0 | 410 | 0.5923 | {'precision': 0.6067892503536068, 'recall': 0.6175623800383877, 'f1-score': 0.612128418549346, 'support': 4168.0} | {'precision': 0.7623089983022071, 'recall': 0.8345724907063197, 'f1-score': 0.7968056787932565, 'support': 2152.0} | {'precision': 0.9368265850062379, 'recall': 0.8952959028831563, 'f1-score': 0.9155905337249902, 'support': 9226.0} | {'precision': 0.8744673877417241, 'recall': 0.8839559347303901, 'f1-score': 0.879186060880669, 'support': 12073.0} | 0.8437 | {'precision': 0.795098055350944, 'recall': 0.8078466770895635, 'f1-score': 0.8009276729870654, 'support': 27619.0} | {'precision': 0.846163633922067, 'recall': 0.8436945580940657, 'f1-score': 0.8446261141401151, 'support': 27619.0} |
83
+ | No log | 11.0 | 451 | 0.6036 | {'precision': 0.5938604240282686, 'recall': 0.6451535508637236, 'f1-score': 0.6184452621895125, 'support': 4168.0} | {'precision': 0.7668161434977578, 'recall': 0.7946096654275093, 'f1-score': 0.7804655408489276, 'support': 2152.0} | {'precision': 0.9390562819783969, 'recall': 0.8951875135486668, 'f1-score': 0.9165973031463293, 'support': 9226.0} | {'precision': 0.8781700646444555, 'recall': 0.8776608962146939, 'f1-score': 0.8779154066034218, 'support': 12073.0} | 0.8420 | {'precision': 0.7944757285372197, 'recall': 0.8031529065136485, 'f1-score': 0.7983558781970478, 'support': 27619.0} | {'precision': 0.8469270804932185, 'recall': 0.841956624063145, 'f1-score': 0.8440870820617664, 'support': 27619.0} |
84
+ | No log | 12.0 | 492 | 0.6292 | {'precision': 0.594930767425487, 'recall': 0.6082053742802304, 'f1-score': 0.6014948392454621, 'support': 4168.0} | {'precision': 0.7890961262553802, 'recall': 0.766728624535316, 'f1-score': 0.7777515908555267, 'support': 2152.0} | {'precision': 0.9292805354155047, 'recall': 0.9029915456319099, 'f1-score': 0.9159474465394976, 'support': 9226.0} | {'precision': 0.872541050235734, 'recall': 0.8890913608879317, 'f1-score': 0.8807384615384615, 'support': 12073.0} | 0.8418 | {'precision': 0.7964621198330264, 'recall': 0.791754226333847, 'f1-score': 0.7939830845447369, 'support': 27619.0} | {'precision': 0.8430984692266364, 'recall': 0.8418117962272349, 'f1-score': 0.8423345704559697, 'support': 27619.0} |
85
+ | 0.2689 | 13.0 | 533 | 0.6506 | {'precision': 0.6016401590457257, 'recall': 0.5808541266794626, 'f1-score': 0.5910644531250001, 'support': 4168.0} | {'precision': 0.7968977217644208, 'recall': 0.7639405204460966, 'f1-score': 0.7800711743772243, 'support': 2152.0} | {'precision': 0.9178990865593737, 'recall': 0.9149143724257534, 'f1-score': 0.9164042992074695, 'support': 9226.0} | {'precision': 0.8670557717250325, 'recall': 0.8859438416300837, 'f1-score': 0.8763980498996273, 'support': 12073.0} | 0.8401 | {'precision': 0.7958731847736381, 'recall': 0.786413215295349, 'f1-score': 0.7909844941523303, 'support': 27619.0} | {'precision': 0.8385191855162285, 'recall': 0.8400738621963141, 'f1-score': 0.8391965505199718, 'support': 27619.0} |
86
+ | 0.2689 | 14.0 | 574 | 0.6476 | {'precision': 0.6124620060790273, 'recall': 0.5801343570057581, 'f1-score': 0.5958600295712174, 'support': 4168.0} | {'precision': 0.77728285077951, 'recall': 0.8108736059479554, 'f1-score': 0.7937229929497385, 'support': 2152.0} | {'precision': 0.9248128577719067, 'recall': 0.9105787990461739, 'f1-score': 0.9176406335335883, 'support': 9226.0} | {'precision': 0.8699562469615946, 'recall': 0.8893398492503934, 'f1-score': 0.8795412656154004, 'support': 12073.0} | 0.8437 | {'precision': 0.7961284903980096, 'recall': 0.7977316528125702, 'f1-score': 0.7966912304174861, 'support': 27619.0} | {'precision': 0.8422013661459805, 'recall': 0.8436583511350881, 'f1-score': 0.8427709427870771, 'support': 27619.0} |
87
+ | 0.2689 | 15.0 | 615 | 0.6652 | {'precision': 0.5992348158775705, 'recall': 0.6012476007677543, 'f1-score': 0.6002395209580837, 'support': 4168.0} | {'precision': 0.7798372513562387, 'recall': 0.8015799256505576, 'f1-score': 0.7905591200733272, 'support': 2152.0} | {'precision': 0.9388219240391176, 'recall': 0.8948623455451984, 'f1-score': 0.916315205327414, 'support': 9226.0} | {'precision': 0.8667846512750382, 'recall': 0.8924873685082415, 'f1-score': 0.8794482533463925, 'support': 12073.0} | 0.8422 | {'precision': 0.7961696606369912, 'recall': 0.797544310117938, 'f1-score': 0.7966405249263044, 'support': 27619.0} | {'precision': 0.8436975503647769, 'recall': 0.842246279734965, 'f1-score': 0.842701922471951, 'support': 27619.0} |
88
+ | 0.2689 | 16.0 | 656 | 0.6609 | {'precision': 0.6078710289236605, 'recall': 0.6151631477927063, 'f1-score': 0.6114953493918436, 'support': 4168.0} | {'precision': 0.782967032967033, 'recall': 0.7946096654275093, 'f1-score': 0.7887453874538746, 'support': 2152.0} | {'precision': 0.934072084172823, 'recall': 0.9045089963147627, 'f1-score': 0.9190528634361235, 'support': 9226.0} | {'precision': 0.8725067166001791, 'recall': 0.8876832601673155, 'f1-score': 0.8800295615043522, 'support': 12073.0} | 0.8449 | {'precision': 0.7993542156659239, 'recall': 0.8004912674255735, 'f1-score': 0.7998307904465485, 'support': 27619.0} | {'precision': 0.8461593157460915, 'recall': 0.8449255946993012, 'f1-score': 0.8454278324403368, 'support': 27619.0} |
89
 
90
 
91
  ### Framework versions
meta_data/meta_s42_e16_cvi4.json CHANGED
@@ -1 +1 @@
1
- {"Claim": {"precision": 0.5992348158775705, "recall": 0.6012476007677543, "f1-score": 0.6002395209580837, "support": 4168.0}, "MajorClaim": {"precision": 0.7798372513562387, "recall": 0.8015799256505576, "f1-score": 0.7905591200733272, "support": 2152.0}, "O": {"precision": 0.9388219240391176, "recall": 0.8948623455451984, "f1-score": 0.916315205327414, "support": 9226.0}, "Premise": {"precision": 0.8667846512750382, "recall": 0.8924873685082415, "f1-score": 0.8794482533463925, "support": 12073.0}, "accuracy": 0.842246279734965, "macro avg": {"precision": 0.7961696606369912, "recall": 0.797544310117938, "f1-score": 0.7966405249263044, "support": 27619.0}, "weighted avg": {"precision": 0.8436975503647769, "recall": 0.842246279734965, "f1-score": 0.842701922471951, "support": 27619.0}}
 
1
+ {"Claim": {"precision": 0.6078710289236605, "recall": 0.6151631477927063, "f1-score": 0.6114953493918436, "support": 4168.0}, "MajorClaim": {"precision": 0.782967032967033, "recall": 0.7946096654275093, "f1-score": 0.7887453874538746, "support": 2152.0}, "O": {"precision": 0.934072084172823, "recall": 0.9045089963147627, "f1-score": 0.9190528634361235, "support": 9226.0}, "Premise": {"precision": 0.8725067166001791, "recall": 0.8876832601673155, "f1-score": 0.8800295615043522, "support": 12073.0}, "accuracy": 0.8449255946993012, "macro avg": {"precision": 0.7993542156659239, "recall": 0.8004912674255735, "f1-score": 0.7998307904465485, "support": 27619.0}, "weighted avg": {"precision": 0.8461593157460915, "recall": 0.8449255946993012, "f1-score": 0.8454278324403368, "support": 27619.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a69e6ea5750d150ff1dd4ce0fc02f96f3acc47be838d2c909f35f2ad669195b9
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d598871e4c8f3102f669a22583c163485acc2797fd68fdb87af59383d4013a22
3
  size 592324828