trainer: training complete at 2024-02-19 19:40:29.678854.

README.md

@@ -22,7 +22,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.9400193485972267
+      value: 0.9360779676806765
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,13 +32,13 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1716
-- B: {'precision': 0.8278829604130808, 'recall': 0.9084041548630784, 'f1-score': 0.8662764520486267, 'support': 1059.0}
-- I: {'precision': 0.949054915557544, 'recall': 0.9656330014224751, 'f1-score': 0.9572721888484643, 'support': 17575.0}
-- O: {'precision': 0.9364918217710095, 'recall': 0.8950943396226415, 'f1-score': 0.9153252480705623, 'support': 9275.0}
-- Accuracy: 0.9400
-- Macro avg: {'precision': 0.9044765659138781, 'recall': 0.9230438319693982, 'f1-score': 0.9129579629892177, 'support': 27909.0}
-- Weighted avg: {'precision': 0.9402819822611845, 'recall': 0.9400193485972267, 'f1-score': 0.9398791485752167, 'support': 27909.0}
+- Loss: 0.1786
+- B: {'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0}
+- I: {'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0}
+- O: {'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0}
+- Accuracy: 0.9361
+- Macro avg: {'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}
+- Weighted avg: {'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0}
 
 ## Model description
 
@@ -63,17 +63,16 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 5
+- num_epochs: 4
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | B                                                                                                                  | I                                                                                                                   | O                                                                                                                  | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
 |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.2820          | {'precision': 0.8252595155709342, 'recall': 0.45042492917847027, 'f1-score': 0.582773365913256, 'support': 1059.0} | {'precision': 0.9113681210260908, 'recall': 0.9460597439544808, 'f1-score': 0.9283899606354169, 'support': 17575.0} | {'precision': 0.879608231539562, 'recall': 0.8617789757412398, 'f1-score': 0.8706023309007732, 'support': 9275.0}  | 0.8992   | {'precision': 0.8720786227121957, 'recall': 0.7527545496247302, 'f1-score': 0.793921885816482, 'support': 27909.0}  | {'precision': 0.8975459852217064, 'recall': 0.8992439714787345, 'f1-score': 0.896071058503503, 'support': 27909.0}  |
-| No log        | 2.0   | 82   | 0.1953          | {'precision': 0.812897366030881, 'recall': 0.8451369216241738, 'f1-score': 0.8287037037037038, 'support': 1059.0}  | {'precision': 0.9452124358178637, 'recall': 0.9531721194879089, 'f1-score': 0.9491755906850247, 'support': 17575.0} | {'precision': 0.9121629058888278, 'recall': 0.8934770889487871, 'f1-score': 0.902723311546841, 'support': 9275.0}  | 0.9292   | {'precision': 0.8900909025791909, 'recall': 0.8972620433536234, 'f1-score': 0.8935342019785232, 'support': 27909.0} | {'precision': 0.9292084210199053, 'recall': 0.9292342971801211, 'f1-score': 0.9291668258665118, 'support': 27909.0} |
-| No log        | 3.0   | 123  | 0.1858          | {'precision': 0.7883211678832117, 'recall': 0.9178470254957507, 'f1-score': 0.8481675392670156, 'support': 1059.0} | {'precision': 0.9373831775700935, 'recall': 0.9701849217638692, 'f1-score': 0.9535020271214875, 'support': 17575.0} | {'precision': 0.9481498939429649, 'recall': 0.8674932614555256, 'f1-score': 0.9060300658746692, 'support': 9275.0} | 0.9341   | {'precision': 0.8912847464654234, 'recall': 0.9185084029050485, 'f1-score': 0.9025665440877241, 'support': 27909.0} | {'precision': 0.9353051606615684, 'recall': 0.9340714464867964, 'f1-score': 0.9337287760841115, 'support': 27909.0} |
-| No log        | 4.0   | 164  | 0.1704          | {'precision': 0.8296943231441049, 'recall': 0.8970727101038716, 'f1-score': 0.8620689655172413, 'support': 1059.0} | {'precision': 0.9604448520981427, 'recall': 0.9532859174964438, 'f1-score': 0.9568519946314857, 'support': 17575.0} | {'precision': 0.9158798283261803, 'recall': 0.9203234501347709, 'f1-score': 0.9180962624361388, 'support': 9275.0} | 0.9402   | {'precision': 0.9020063345228092, 'recall': 0.923560692578362, 'f1-score': 0.9123390741949553, 'support': 27909.0}  | {'precision': 0.9406732585029842, 'recall': 0.9401985022752517, 'f1-score': 0.9403757810823142, 'support': 27909.0} |
-| No log        | 5.0   | 205  | 0.1716          | {'precision': 0.8278829604130808, 'recall': 0.9084041548630784, 'f1-score': 0.8662764520486267, 'support': 1059.0} | {'precision': 0.949054915557544, 'recall': 0.9656330014224751, 'f1-score': 0.9572721888484643, 'support': 17575.0}  | {'precision': 0.9364918217710095, 'recall': 0.8950943396226415, 'f1-score': 0.9153252480705623, 'support': 9275.0} | 0.9400   | {'precision': 0.9044765659138781, 'recall': 0.9230438319693982, 'f1-score': 0.9129579629892177, 'support': 27909.0} | {'precision': 0.9402819822611845, 'recall': 0.9400193485972267, 'f1-score': 0.9398791485752167, 'support': 27909.0} |
+| No log        | 1.0   | 41   | 0.2836          | {'precision': 0.7968253968253968, 'recall': 0.4740321057601511, 'f1-score': 0.5944345766725874, 'support': 1059.0} | {'precision': 0.9161490683229814, 'recall': 0.9399715504978663, 'f1-score': 0.9279074339315303, 'support': 17575.0} | {'precision': 0.8717421866551314, 'recall': 0.8691105121293801, 'f1-score': 0.8704243602202786, 'support': 9275.0} | 0.8987   | {'precision': 0.8615722172678364, 'recall': 0.7610380561291326, 'f1-score': 0.7975887902747987, 'support': 27909.0} | {'precision': 0.8968636193428943, 'recall': 0.8987423411802644, 'f1-score': 0.8961505359950553, 'support': 27909.0} |
+| No log        | 2.0   | 82   | 0.2022          | {'precision': 0.7857142857142857, 'recall': 0.8621340887629839, 'f1-score': 0.8221521837010356, 'support': 1059.0} | {'precision': 0.9355464420305919, 'recall': 0.9605120910384068, 'f1-score': 0.9478649035627053, 'support': 17575.0} | {'precision': 0.9268068482132598, 'recall': 0.8696495956873316, 'f1-score': 0.8973189453776839, 'support': 9275.0} | 0.9266   | {'precision': 0.8826891919860458, 'recall': 0.8974319251629076, 'f1-score': 0.889112010880475, 'support': 27909.0}  | {'precision': 0.9269566686171867, 'recall': 0.926582822745351, 'f1-score': 0.9262968240005718, 'support': 27909.0}  |
+| No log        | 3.0   | 123  | 0.1852          | {'precision': 0.7871125611745514, 'recall': 0.9112370160528801, 'f1-score': 0.8446389496717724, 'support': 1059.0} | {'precision': 0.9400952275495515, 'recall': 0.966145092460882, 'f1-score': 0.9529421668490613, 'support': 17575.0}  | {'precision': 0.940726133859181, 'recall': 0.8743935309973045, 'f1-score': 0.9063477872150201, 'support': 9275.0}  | 0.9336   | {'precision': 0.8893113075277613, 'recall': 0.9172585465036889, 'f1-score': 0.9013096345786179, 'support': 27909.0} | {'precision': 0.9345000078114988, 'recall': 0.9335698161883264, 'f1-score': 0.9333479148838716, 'support': 27909.0} |
+| No log        | 4.0   | 164  | 0.1786          | {'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0} | {'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0} | {'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0} | 0.9361   | {'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}  | {'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0} |
 
 
 ### Framework versions

meta_data/README_s42_e4.md

@@ -0,0 +1,83 @@
+---
+license: apache-2.0
+base_model: allenai/longformer-base-4096
+tags:
+- generated_from_trainer
+datasets:
+- essays_su_g
+metrics:
+- accuracy
+model-index:
+- name: longformer-spans
+  results:
+  - task:
+      name: Token Classification
+      type: token-classification
+    dataset:
+      name: essays_su_g
+      type: essays_su_g
+      config: spans
+      split: test
+      args: spans
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9360779676806765
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# longformer-spans
+
+This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1786
+- B: {'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0}
+- I: {'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0}
+- O: {'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0}
+- Accuracy: 0.9361
+- Macro avg: {'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}
+- Weighted avg: {'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0}
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | B                                                                                                                  | I                                                                                                                   | O                                                                                                                  | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.2836          | {'precision': 0.7968253968253968, 'recall': 0.4740321057601511, 'f1-score': 0.5944345766725874, 'support': 1059.0} | {'precision': 0.9161490683229814, 'recall': 0.9399715504978663, 'f1-score': 0.9279074339315303, 'support': 17575.0} | {'precision': 0.8717421866551314, 'recall': 0.8691105121293801, 'f1-score': 0.8704243602202786, 'support': 9275.0} | 0.8987   | {'precision': 0.8615722172678364, 'recall': 0.7610380561291326, 'f1-score': 0.7975887902747987, 'support': 27909.0} | {'precision': 0.8968636193428943, 'recall': 0.8987423411802644, 'f1-score': 0.8961505359950553, 'support': 27909.0} |
+| No log        | 2.0   | 82   | 0.2022          | {'precision': 0.7857142857142857, 'recall': 0.8621340887629839, 'f1-score': 0.8221521837010356, 'support': 1059.0} | {'precision': 0.9355464420305919, 'recall': 0.9605120910384068, 'f1-score': 0.9478649035627053, 'support': 17575.0} | {'precision': 0.9268068482132598, 'recall': 0.8696495956873316, 'f1-score': 0.8973189453776839, 'support': 9275.0} | 0.9266   | {'precision': 0.8826891919860458, 'recall': 0.8974319251629076, 'f1-score': 0.889112010880475, 'support': 27909.0}  | {'precision': 0.9269566686171867, 'recall': 0.926582822745351, 'f1-score': 0.9262968240005718, 'support': 27909.0}  |
+| No log        | 3.0   | 123  | 0.1852          | {'precision': 0.7871125611745514, 'recall': 0.9112370160528801, 'f1-score': 0.8446389496717724, 'support': 1059.0} | {'precision': 0.9400952275495515, 'recall': 0.966145092460882, 'f1-score': 0.9529421668490613, 'support': 17575.0}  | {'precision': 0.940726133859181, 'recall': 0.8743935309973045, 'f1-score': 0.9063477872150201, 'support': 9275.0}  | 0.9336   | {'precision': 0.8893113075277613, 'recall': 0.9172585465036889, 'f1-score': 0.9013096345786179, 'support': 27909.0} | {'precision': 0.9345000078114988, 'recall': 0.9335698161883264, 'f1-score': 0.9333479148838716, 'support': 27909.0} |
+| No log        | 4.0   | 164  | 0.1786          | {'precision': 0.8126064735945485, 'recall': 0.9008498583569405, 'f1-score': 0.8544558889386475, 'support': 1059.0} | {'precision': 0.9468377121729875, 'recall': 0.9617069701280228, 'f1-score': 0.9542144187884605, 'support': 17575.0} | {'precision': 0.9307744259342638, 'recall': 0.8915363881401617, 'f1-score': 0.9107329698771959, 'support': 9275.0} | 0.9361   | {'precision': 0.8967395372339334, 'recall': 0.918031072208375, 'f1-score': 0.9064677592014346, 'support': 27909.0}  | {'precision': 0.9364060284323041, 'recall': 0.9360779676806765, 'f1-score': 0.9359789133327677, 'support': 27909.0} |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.2.0+cu121
+- Datasets 2.17.0
+- Tokenizers 0.15.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:03cc7a3ac89368fc88aab2bbcc09d669506c0956cec0d49cc552c0fdd4cf55ce
+oid sha256:031119bd36583c8375c8fb54e4564b86ebeb57c9711fec789f96a27378b591c9
 size 592318676