WITHINAPPS_NDD-pagekit_test-content-LongFormer
This model is a fine-tuned version of allenai/longformer-base-4096 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.1262
- Accuracy: 0.8643
- F1: 0.8651
- Precision: 0.8941
- Recall: 0.8643
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
---|---|---|---|---|---|---|---|
0.1837 | 1.0 | 973 | 0.1281 | 0.8469 | 0.8474 | 0.8841 | 0.8469 |
0.1508 | 2.0 | 1946 | 0.1247 | 0.8561 | 0.8571 | 0.8792 | 0.8561 |
0.1374 | 3.0 | 2919 | 0.1459 | 0.8571 | 0.8581 | 0.8773 | 0.8571 |
0.1353 | 4.0 | 3892 | 0.1400 | 0.8607 | 0.8616 | 0.8860 | 0.8607 |
0.1313 | 5.0 | 4865 | 0.1416 | 0.8587 | 0.8596 | 0.8831 | 0.8587 |
0.1281 | 6.0 | 5838 | 0.1195 | 0.8643 | 0.8652 | 0.8907 | 0.8643 |
0.1246 | 7.0 | 6811 | 0.1373 | 0.8623 | 0.8632 | 0.8866 | 0.8623 |
0.1261 | 8.0 | 7784 | 0.1193 | 0.8618 | 0.8626 | 0.8895 | 0.8618 |
0.1168 | 9.0 | 8757 | 0.1223 | 0.8643 | 0.8650 | 0.8955 | 0.8643 |
0.1184 | 10.0 | 9730 | 0.1232 | 0.8638 | 0.8648 | 0.8817 | 0.8638 |
0.1195 | 11.0 | 10703 | 0.1192 | 0.8664 | 0.8671 | 0.8971 | 0.8664 |
0.1176 | 12.0 | 11676 | 0.1294 | 0.8654 | 0.8661 | 0.8956 | 0.8654 |
0.1126 | 13.0 | 12649 | 0.1199 | 0.8602 | 0.8612 | 0.8766 | 0.8602 |
0.115 | 14.0 | 13622 | 0.1200 | 0.8633 | 0.8641 | 0.8931 | 0.8633 |
0.1102 | 15.0 | 14595 | 0.1213 | 0.8654 | 0.8661 | 0.8961 | 0.8654 |
0.1089 | 16.0 | 15568 | 0.1229 | 0.8628 | 0.8636 | 0.8902 | 0.8628 |
0.112 | 17.0 | 16541 | 0.1199 | 0.8669 | 0.8676 | 0.8983 | 0.8669 |
0.1096 | 18.0 | 17514 | 0.1242 | 0.8669 | 0.8676 | 0.8983 | 0.8669 |
0.1091 | 19.0 | 18487 | 0.1262 | 0.8643 | 0.8651 | 0.8941 | 0.8643 |
0.108 | 20.0 | 19460 | 0.1262 | 0.8643 | 0.8651 | 0.8941 | 0.8643 |
Framework versions
- Transformers 4.44.0
- Pytorch 2.4.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
- Downloads last month
- 0
Model tree for lgk03/WITHINAPPS_NDD-pagekit_test-content-LongFormer
Base model
allenai/longformer-base-4096