trainer: training complete at 2024-01-28 12:35:39.089322.

README.md

@@ -22,7 +22,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.8056899208140743
+      value: 0.8069798272958544
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,17 +32,17 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the fancy_dataset dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5149
-- B-claim: {'precision': 1.0, 'recall': 0.0036101083032490976, 'f1-score': 0.007194244604316546, 'support': 277.0}
+- Loss: 0.5164
+- B-claim: {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0}
 - B-majorclaim: {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}
-- B-premise: {'precision': 0.5939759036144578, 'recall': 0.7691107644305772, 'f1-score': 0.6702923181509177, 'support': 641.0}
-- I-claim: {'precision': 0.5709936340990867, 'recall': 0.5057612159843099, 'f1-score': 0.5364014560582424, 'support': 4079.0}
-- I-majorclaim: {'precision': 0.6075593084036992, 'recall': 0.7403233708966193, 'f1-score': 0.6674028268551236, 'support': 2041.0}
-- I-premise: {'precision': 0.8469945355191257, 'recall': 0.8930597992143169, 'f1-score': 0.8694174138443888, 'support': 11455.0}
-- O: {'precision': 0.92, 'recall': 0.8828032345013477, 'f1-score': 0.9010178817056397, 'support': 9275.0}
-- Accuracy: 0.8057
-- Macro avg: {'precision': 0.6485033402337671, 'recall': 0.5420954990472029, 'f1-score': 0.5216751630312327, 'support': 27909.0}
-- Weighted avg: {'precision': 0.8048361654136865, 'recall': 0.8056899208140743, 'f1-score': 0.7989508122458809, 'support': 27909.0}
+- B-premise: {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0}
+- I-claim: {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0}
+- I-majorclaim: {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0}
+- I-premise: {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0}
+- O: {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0}
+- Accuracy: 0.8070
+- Macro avg: {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0}
+- Weighted avg: {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0}
 
 ## Model description
 
@@ -71,11 +71,11 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                 | B-majorclaim                                                         | B-premise                                                                                                         | I-claim                                                                                                             | I-majorclaim                                                                                                           | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                             | Weighted avg                                                                                                        |
-|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:---------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.7274          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                    | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.78, 'recall': 0.060842433697347896, 'f1-score': 0.11287988422575977, 'support': 641.0}            | {'precision': 0.36033007334963324, 'recall': 0.28904143172346164, 'f1-score': 0.320772683988573, 'support': 4079.0} | {'precision': 0.5909090909090909, 'recall': 0.012738853503184714, 'f1-score': 0.024940047961630695, 'support': 2041.0} | {'precision': 0.7395335962909141, 'recall': 0.9329550414666085, 'f1-score': 0.8250598316992201, 'support': 11455.0} | {'precision': 0.8197582243361078, 'recall': 0.8919676549865229, 'f1-score': 0.8543398564568596, 'support': 9275.0} | 0.7239   | {'precision': 0.47007585498367804, 'recall': 0.3125064879110179, 'f1-score': 0.30542747204743476, 'support': 27909.0} | {'precision': 0.6897569493700394, 'recall': 0.7239241821634598, 'f1-score': 0.6738597929850489, 'support': 27909.0} |
-| No log        | 2.0   | 82   | 0.5478          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                    | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6131687242798354, 'recall': 0.6973478939157566, 'f1-score': 0.6525547445255474, 'support': 641.0} | {'precision': 0.553030303030303, 'recall': 0.4295170384898259, 'f1-score': 0.48351041810404305, 'support': 4079.0}  | {'precision': 0.6469656992084433, 'recall': 0.600685938265556, 'f1-score': 0.6229674796747968, 'support': 2041.0}      | {'precision': 0.8183098591549296, 'recall': 0.9129637712789175, 'f1-score': 0.8630493088508355, 'support': 11455.0} | {'precision': 0.8851879618721217, 'recall': 0.8911051212938005, 'f1-score': 0.8881366860090264, 'support': 9275.0} | 0.7936   | {'precision': 0.5023803639350904, 'recall': 0.5045171090348366, 'f1-score': 0.5014598053091784, 'support': 27909.0}   | {'precision': 0.7722658115085478, 'recall': 0.7935791321795836, 'f1-score': 0.7805976856327196, 'support': 27909.0} |
-| No log        | 3.0   | 123  | 0.5149          | {'precision': 1.0, 'recall': 0.0036101083032490976, 'f1-score': 0.007194244604316546, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.5939759036144578, 'recall': 0.7691107644305772, 'f1-score': 0.6702923181509177, 'support': 641.0} | {'precision': 0.5709936340990867, 'recall': 0.5057612159843099, 'f1-score': 0.5364014560582424, 'support': 4079.0}  | {'precision': 0.6075593084036992, 'recall': 0.7403233708966193, 'f1-score': 0.6674028268551236, 'support': 2041.0}     | {'precision': 0.8469945355191257, 'recall': 0.8930597992143169, 'f1-score': 0.8694174138443888, 'support': 11455.0} | {'precision': 0.92, 'recall': 0.8828032345013477, 'f1-score': 0.9010178817056397, 'support': 9275.0}               | 0.8057   | {'precision': 0.6485033402337671, 'recall': 0.5420954990472029, 'f1-score': 0.5216751630312327, 'support': 27909.0}   | {'precision': 0.8048361654136865, 'recall': 0.8056899208140743, 'f1-score': 0.7989508122458809, 'support': 27909.0} |
+| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                               | B-majorclaim                                                         | B-premise                                                                                                           | I-claim                                                                                                               | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                            | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.7242          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                  | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.8028169014084507, 'recall': 0.08892355694227769, 'f1-score': 0.16011235955056177, 'support': 641.0} | {'precision': 0.43010291595197253, 'recall': 0.24589360137288552, 'f1-score': 0.31289970363437836, 'support': 4079.0} | {'precision': 0.6835106382978723, 'recall': 0.1259186673199412, 'f1-score': 0.2126603227141084, 'support': 2041.0} | {'precision': 0.7517079419299744, 'recall': 0.9221300742034046, 'f1-score': 0.8282432273493551, 'support': 11455.0} | {'precision': 0.7629536017331648, 'recall': 0.9112668463611859, 'f1-score': 0.8305409521937798, 'support': 9275.0} | 0.7285   | {'precision': 0.49015599990306213, 'recall': 0.3277332494570993, 'f1-score': 0.3349223664917405, 'support': 27909.0} | {'precision': 0.6933695141932649, 'recall': 0.7285105163209, 'f1-score': 0.6809195289383426, 'support': 27909.0}    |
+| No log        | 2.0   | 82   | 0.5451          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                  | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.638235294117647, 'recall': 0.6770670826833073, 'f1-score': 0.6570779712339135, 'support': 641.0}    | {'precision': 0.5605615409729023, 'recall': 0.4209365040451091, 'f1-score': 0.48081769812377484, 'support': 4079.0}   | {'precision': 0.6609442060085837, 'recall': 0.6036256736893679, 'f1-score': 0.6309859154929578, 'support': 2041.0} | {'precision': 0.8157935644333904, 'recall': 0.916281099956351, 'f1-score': 0.8631224045063938, 'support': 11455.0}  | {'precision': 0.8817295464179737, 'recall': 0.8970350404312668, 'f1-score': 0.8893164448720005, 'support': 9275.0} | 0.7954   | {'precision': 0.5081805931357853, 'recall': 0.5021350572579146, 'f1-score': 0.5030457763184344, 'support': 27909.0}  | {'precision': 0.7727823747619976, 'recall': 0.7954064996954388, 'f1-score': 0.7813164854898953, 'support': 27909.0} |
+| No log        | 3.0   | 123  | 0.5164          | {'precision': 0.5, 'recall': 0.01444043321299639, 'f1-score': 0.028070175438596492, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6012121212121212, 'recall': 0.7737909516380655, 'f1-score': 0.6766712141882674, 'support': 641.0}   | {'precision': 0.5778401122019635, 'recall': 0.5050257416033341, 'f1-score': 0.5389848246991105, 'support': 4079.0}    | {'precision': 0.6294978252273626, 'recall': 0.7800097991180793, 'f1-score': 0.6967177242888402, 'support': 2041.0} | {'precision': 0.8417716308553552, 'recall': 0.8926233085988651, 'f1-score': 0.8664519955935939, 'support': 11455.0} | {'precision': 0.9219015280135824, 'recall': 0.8781671159029649, 'f1-score': 0.8995030369961348, 'support': 9275.0} | 0.8070   | {'precision': 0.581746173930055, 'recall': 0.549151050010615, 'f1-score': 0.5294855673149347, 'support': 27909.0}    | {'precision': 0.8011330593153426, 'recall': 0.8069798272958544, 'f1-score': 0.8001053402048132, 'support': 27909.0} |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4586c95aa9bce865d4c7ff6cbbb0fd2c4cc5dd4ca370581fcc8b60ae6fedf25c
+oid sha256:dd0b152b04957f2b0f66ca6789a614173447837debbcebedc2eb929341de7383
 size 592330980