Hugging Face's logo

DiegoBraz
/
swin-tiny-patch4-window7-224-finetuned-eurosat

Image Classification
Transformers
TensorBoard
Safetensors
swin
Generated from Trainer
Eval Results
Model card Files
xet
Metrics Community
DiegoBraz commited on
Commit
4febf8d
·
verified ·
1 Parent(s): fdb2063

Training in progress, epoch 0

Browse files
Files changed (12) hide show
  1. README.md +124 -0
  2. all_results.json +13 -0
  3. config.json +79 -0
  4. eval_results.json +8 -0
  5. model.safetensors +3 -0
  6. preprocessor_config.json +22 -0
  7. runs/Sep19_06-34-28_6ebaceb369b5/events.out.tfevents.1726727670.6ebaceb369b5.376.0 +3 -0
  8. runs/Sep19_06-34-28_6ebaceb369b5/events.out.tfevents.1726729758.6ebaceb369b5.376.1 +3 -0
  9. runs/Sep19_07-20-52_6ebaceb369b5/events.out.tfevents.1726730454.6ebaceb369b5.376.2 +3 -0
  10. train_results.json +8 -0
  11. trainer_state.json +701 -0
  12. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,124 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: microsoft/swin-tiny-patch4-window7-224
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - imagefolder
9
+ metrics:
10
+ - accuracy
11
+ model-index:
12
+ - name: swin-tiny-patch4-window7-224-finetuned-eurosat
13
+ results:
14
+ - task:
15
+ name: Image Classification
16
+ type: image-classification
17
+ dataset:
18
+ name: imagefolder
19
+ type: imagefolder
20
+ config: default
21
+ split: train
22
+ args: default
23
+ metrics:
24
+ - name: Accuracy
25
+ type: accuracy
26
+ value: 0.8909090909090909
27
+ ---
28
+
29
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
30
+ should probably proofread and complete it, then remove this comment. -->
31
+
32
+ # swin-tiny-patch4-window7-224-finetuned-eurosat
33
+
34
+ This model is a fine-tuned version of [microsoft/swin-tiny-patch4-window7-224](https://huggingface.co/microsoft/swin-tiny-patch4-window7-224) on the imagefolder dataset.
35
+ It achieves the following results on the evaluation set:
36
+ - Loss: 0.3813
37
+ - Accuracy: 0.8909
38
+
39
+ ## Model description
40
+
41
+ More information needed
42
+
43
+ ## Intended uses & limitations
44
+
45
+ More information needed
46
+
47
+ ## Training and evaluation data
48
+
49
+ More information needed
50
+
51
+ ## Training procedure
52
+
53
+ ### Training hyperparameters
54
+
55
+ The following hyperparameters were used during training:
56
+ - learning_rate: 5e-05
57
+ - train_batch_size: 32
58
+ - eval_batch_size: 32
59
+ - seed: 42
60
+ - gradient_accumulation_steps: 4
61
+ - total_train_batch_size: 128
62
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
63
+ - lr_scheduler_type: linear
64
+ - lr_scheduler_warmup_ratio: 0.1
65
+ - num_epochs: 50
66
+
67
+ ### Training results
68
+
69
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
70
+ |:-------------:|:-------:|:----:|:---------------:|:--------:|
71
+ | No log | 0.9032 | 7 | 2.3655 | 0.1455 |
72
+ | 2.396 | 1.9355 | 15 | 2.2806 | 0.2 |
73
+ | 2.3064 | 2.9677 | 23 | 2.1057 | 0.3727 |
74
+ | 2.0698 | 4.0 | 31 | 1.7731 | 0.5636 |
75
+ | 2.0698 | 4.9032 | 38 | 1.3060 | 0.6182 |
76
+ | 1.5736 | 5.9355 | 46 | 0.8939 | 0.7182 |
77
+ | 0.9943 | 6.9677 | 54 | 0.7154 | 0.7909 |
78
+ | 0.8023 | 8.0 | 62 | 0.6640 | 0.7727 |
79
+ | 0.8023 | 8.9032 | 69 | 0.5833 | 0.7818 |
80
+ | 0.5882 | 9.9355 | 77 | 0.5443 | 0.8091 |
81
+ | 0.5332 | 10.9677 | 85 | 0.5864 | 0.7909 |
82
+ | 0.4483 | 12.0 | 93 | 0.4938 | 0.8273 |
83
+ | 0.378 | 12.9032 | 100 | 0.4696 | 0.8364 |
84
+ | 0.378 | 13.9355 | 108 | 0.4419 | 0.8545 |
85
+ | 0.3461 | 14.9677 | 116 | 0.4350 | 0.8636 |
86
+ | 0.333 | 16.0 | 124 | 0.4285 | 0.8727 |
87
+ | 0.2771 | 16.9032 | 131 | 0.4151 | 0.8636 |
88
+ | 0.2771 | 17.9355 | 139 | 0.3938 | 0.8818 |
89
+ | 0.2791 | 18.9677 | 147 | 0.3853 | 0.8818 |
90
+ | 0.2939 | 20.0 | 155 | 0.4061 | 0.8636 |
91
+ | 0.2651 | 20.9032 | 162 | 0.4434 | 0.8545 |
92
+ | 0.2462 | 21.9355 | 170 | 0.3813 | 0.8909 |
93
+ | 0.2462 | 22.9677 | 178 | 0.4007 | 0.8818 |
94
+ | 0.2277 | 24.0 | 186 | 0.3784 | 0.8727 |
95
+ | 0.2289 | 24.9032 | 193 | 0.3682 | 0.8636 |
96
+ | 0.2518 | 25.9355 | 201 | 0.4235 | 0.8636 |
97
+ | 0.2518 | 26.9677 | 209 | 0.4013 | 0.8727 |
98
+ | 0.1961 | 28.0 | 217 | 0.3705 | 0.8727 |
99
+ | 0.2316 | 28.9032 | 224 | 0.3901 | 0.8727 |
100
+ | 0.1802 | 29.9355 | 232 | 0.4017 | 0.8636 |
101
+ | 0.1711 | 30.9677 | 240 | 0.4080 | 0.8455 |
102
+ | 0.1711 | 32.0 | 248 | 0.3773 | 0.8636 |
103
+ | 0.1885 | 32.9032 | 255 | 0.3669 | 0.8727 |
104
+ | 0.1784 | 33.9355 | 263 | 0.4084 | 0.8636 |
105
+ | 0.18 | 34.9677 | 271 | 0.4206 | 0.8636 |
106
+ | 0.18 | 36.0 | 279 | 0.4106 | 0.8636 |
107
+ | 0.1752 | 36.9032 | 286 | 0.4133 | 0.8727 |
108
+ | 0.1778 | 37.9355 | 294 | 0.4184 | 0.8727 |
109
+ | 0.1633 | 38.9677 | 302 | 0.4236 | 0.8636 |
110
+ | 0.1621 | 40.0 | 310 | 0.4168 | 0.8727 |
111
+ | 0.1621 | 40.9032 | 317 | 0.4187 | 0.8727 |
112
+ | 0.1497 | 41.9355 | 325 | 0.4140 | 0.8727 |
113
+ | 0.1434 | 42.9677 | 333 | 0.4118 | 0.8909 |
114
+ | 0.1802 | 44.0 | 341 | 0.4125 | 0.8818 |
115
+ | 0.1802 | 44.9032 | 348 | 0.4124 | 0.8727 |
116
+ | 0.1576 | 45.1613 | 350 | 0.4122 | 0.8727 |
117
+
118
+
119
+ ### Framework versions
120
+
121
+ - Transformers 4.44.2
122
+ - Pytorch 2.4.0+cu121
123
+ - Datasets 3.0.0
124
+ - Tokenizers 0.19.1
all_results.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 45.16129032258065,
3
+ "eval_accuracy": 0.8909090909090909,
4
+ "eval_loss": 0.3812616765499115,
5
+ "eval_runtime": 3.2137,
6
+ "eval_samples_per_second": 34.229,
7
+ "eval_steps_per_second": 1.245,
8
+ "total_flos": 1.1015215348522291e+18,
9
+ "train_loss": 0.49926152706146243,
10
+ "train_runtime": 2076.2527,
11
+ "train_samples_per_second": 23.624,
12
+ "train_steps_per_second": 0.169
13
+ }
config.json ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "microsoft/swin-tiny-patch4-window7-224",
3
+ "architectures": [
4
+ "SwinForImageClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.0,
7
+ "depths": [
8
+ 2,
9
+ 2,
10
+ 6,
11
+ 2
12
+ ],
13
+ "drop_path_rate": 0.1,
14
+ "embed_dim": 96,
15
+ "encoder_stride": 32,
16
+ "hidden_act": "gelu",
17
+ "hidden_dropout_prob": 0.0,
18
+ "hidden_size": 768,
19
+ "id2label": {
20
+ "0": "AK-47",
21
+ "1": "AWP",
22
+ "2": "Famas",
23
+ "3": "Galil-AR",
24
+ "4": "Glock",
25
+ "5": "M4A1",
26
+ "6": "M4A4",
27
+ "7": "P-90",
28
+ "8": "SG-553",
29
+ "9": "UMP",
30
+ "10": "USP"
31
+ },
32
+ "image_size": 224,
33
+ "initializer_range": 0.02,
34
+ "label2id": {
35
+ "AK-47": 0,
36
+ "AWP": 1,
37
+ "Famas": 2,
38
+ "Galil-AR": 3,
39
+ "Glock": 4,
40
+ "M4A1": 5,
41
+ "M4A4": 6,
42
+ "P-90": 7,
43
+ "SG-553": 8,
44
+ "UMP": 9,
45
+ "USP": 10
46
+ },
47
+ "layer_norm_eps": 1e-05,
48
+ "mlp_ratio": 4.0,
49
+ "model_type": "swin",
50
+ "num_channels": 3,
51
+ "num_heads": [
52
+ 3,
53
+ 6,
54
+ 12,
55
+ 24
56
+ ],
57
+ "num_layers": 4,
58
+ "out_features": [
59
+ "stage4"
60
+ ],
61
+ "out_indices": [
62
+ 4
63
+ ],
64
+ "patch_size": 4,
65
+ "path_norm": true,
66
+ "problem_type": "single_label_classification",
67
+ "qkv_bias": true,
68
+ "stage_names": [
69
+ "stem",
70
+ "stage1",
71
+ "stage2",
72
+ "stage3",
73
+ "stage4"
74
+ ],
75
+ "torch_dtype": "float32",
76
+ "transformers_version": "4.44.2",
77
+ "use_absolute_embeddings": false,
78
+ "window_size": 7
79
+ }
eval_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 45.16129032258065,
3
+ "eval_accuracy": 0.8909090909090909,
4
+ "eval_loss": 0.3812616765499115,
5
+ "eval_runtime": 3.2137,
6
+ "eval_samples_per_second": 34.229,
7
+ "eval_steps_per_second": 1.245
8
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7571d68c84fc8b6138f06a14d3f97daf7b238cefc65da4cc0025b676e1eaa987
3
+ size 110370524
preprocessor_config.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "do_rescale": true,
4
+ "do_resize": true,
5
+ "image_mean": [
6
+ 0.485,
7
+ 0.456,
8
+ 0.406
9
+ ],
10
+ "image_processor_type": "ViTImageProcessor",
11
+ "image_std": [
12
+ 0.229,
13
+ 0.224,
14
+ 0.225
15
+ ],
16
+ "resample": 3,
17
+ "rescale_factor": 0.00392156862745098,
18
+ "size": {
19
+ "height": 224,
20
+ "width": 224
21
+ }
22
+ }
runs/Sep19_06-34-28_6ebaceb369b5/events.out.tfevents.1726727670.6ebaceb369b5.376.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75275220284f8170a66d2bcbc736130829084258a7a3bb5fbb1ebaa35e6f3bea
3
+ size 28118
runs/Sep19_06-34-28_6ebaceb369b5/events.out.tfevents.1726729758.6ebaceb369b5.376.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e190e59f50f4ad219747391a6f39883a129b2d1f8383a5f564f42756f493916e
3
+ size 411
runs/Sep19_07-20-52_6ebaceb369b5/events.out.tfevents.1726730454.6ebaceb369b5.376.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dbeff909bea159730b366efdf97d48cf9088b4b2ded0d7bec0505039082eadd
3
+ size 5982
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 45.16129032258065,
3
+ "total_flos": 1.1015215348522291e+18,
4
+ "train_loss": 0.49926152706146243,
5
+ "train_runtime": 2076.2527,
6
+ "train_samples_per_second": 23.624,
7
+ "train_steps_per_second": 0.169
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,701 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.8909090909090909,
3
+ "best_model_checkpoint": "swin-tiny-patch4-window7-224-finetuned-eurosat/checkpoint-170",
4
+ "epoch": 45.16129032258065,
5
+ "eval_steps": 500,
6
+ "global_step": 350,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.9032258064516129,
13
+ "eval_accuracy": 0.14545454545454545,
14
+ "eval_loss": 2.365513563156128,
15
+ "eval_runtime": 42.0143,
16
+ "eval_samples_per_second": 2.618,
17
+ "eval_steps_per_second": 0.095,
18
+ "step": 7
19
+ },
20
+ {
21
+ "epoch": 1.2903225806451613,
22
+ "grad_norm": 6.957228183746338,
23
+ "learning_rate": 1.4285714285714285e-05,
24
+ "loss": 2.396,
25
+ "step": 10
26
+ },
27
+ {
28
+ "epoch": 1.935483870967742,
29
+ "eval_accuracy": 0.2,
30
+ "eval_loss": 2.280606985092163,
31
+ "eval_runtime": 2.4197,
32
+ "eval_samples_per_second": 45.46,
33
+ "eval_steps_per_second": 1.653,
34
+ "step": 15
35
+ },
36
+ {
37
+ "epoch": 2.5806451612903225,
38
+ "grad_norm": 5.89718770980835,
39
+ "learning_rate": 2.857142857142857e-05,
40
+ "loss": 2.3064,
41
+ "step": 20
42
+ },
43
+ {
44
+ "epoch": 2.967741935483871,
45
+ "eval_accuracy": 0.37272727272727274,
46
+ "eval_loss": 2.1056878566741943,
47
+ "eval_runtime": 2.3883,
48
+ "eval_samples_per_second": 46.057,
49
+ "eval_steps_per_second": 1.675,
50
+ "step": 23
51
+ },
52
+ {
53
+ "epoch": 3.870967741935484,
54
+ "grad_norm": 10.595786094665527,
55
+ "learning_rate": 4.2857142857142856e-05,
56
+ "loss": 2.0698,
57
+ "step": 30
58
+ },
59
+ {
60
+ "epoch": 4.0,
61
+ "eval_accuracy": 0.5636363636363636,
62
+ "eval_loss": 1.7731393575668335,
63
+ "eval_runtime": 2.7796,
64
+ "eval_samples_per_second": 39.574,
65
+ "eval_steps_per_second": 1.439,
66
+ "step": 31
67
+ },
68
+ {
69
+ "epoch": 4.903225806451613,
70
+ "eval_accuracy": 0.6181818181818182,
71
+ "eval_loss": 1.306030511856079,
72
+ "eval_runtime": 2.3717,
73
+ "eval_samples_per_second": 46.381,
74
+ "eval_steps_per_second": 1.687,
75
+ "step": 38
76
+ },
77
+ {
78
+ "epoch": 5.161290322580645,
79
+ "grad_norm": 10.822553634643555,
80
+ "learning_rate": 4.9206349206349204e-05,
81
+ "loss": 1.5736,
82
+ "step": 40
83
+ },
84
+ {
85
+ "epoch": 5.935483870967742,
86
+ "eval_accuracy": 0.7181818181818181,
87
+ "eval_loss": 0.893871009349823,
88
+ "eval_runtime": 2.467,
89
+ "eval_samples_per_second": 44.589,
90
+ "eval_steps_per_second": 1.621,
91
+ "step": 46
92
+ },
93
+ {
94
+ "epoch": 6.451612903225806,
95
+ "grad_norm": 11.323225975036621,
96
+ "learning_rate": 4.761904761904762e-05,
97
+ "loss": 0.9943,
98
+ "step": 50
99
+ },
100
+ {
101
+ "epoch": 6.967741935483871,
102
+ "eval_accuracy": 0.7909090909090909,
103
+ "eval_loss": 0.7153930068016052,
104
+ "eval_runtime": 2.7519,
105
+ "eval_samples_per_second": 39.972,
106
+ "eval_steps_per_second": 1.454,
107
+ "step": 54
108
+ },
109
+ {
110
+ "epoch": 7.741935483870968,
111
+ "grad_norm": 16.32503890991211,
112
+ "learning_rate": 4.603174603174603e-05,
113
+ "loss": 0.8023,
114
+ "step": 60
115
+ },
116
+ {
117
+ "epoch": 8.0,
118
+ "eval_accuracy": 0.7727272727272727,
119
+ "eval_loss": 0.6640019416809082,
120
+ "eval_runtime": 2.4176,
121
+ "eval_samples_per_second": 45.5,
122
+ "eval_steps_per_second": 1.655,
123
+ "step": 62
124
+ },
125
+ {
126
+ "epoch": 8.903225806451612,
127
+ "eval_accuracy": 0.7818181818181819,
128
+ "eval_loss": 0.5833413004875183,
129
+ "eval_runtime": 2.9975,
130
+ "eval_samples_per_second": 36.698,
131
+ "eval_steps_per_second": 1.334,
132
+ "step": 69
133
+ },
134
+ {
135
+ "epoch": 9.03225806451613,
136
+ "grad_norm": 8.37897777557373,
137
+ "learning_rate": 4.4444444444444447e-05,
138
+ "loss": 0.5882,
139
+ "step": 70
140
+ },
141
+ {
142
+ "epoch": 9.935483870967742,
143
+ "eval_accuracy": 0.8090909090909091,
144
+ "eval_loss": 0.5442659854888916,
145
+ "eval_runtime": 2.3662,
146
+ "eval_samples_per_second": 46.488,
147
+ "eval_steps_per_second": 1.69,
148
+ "step": 77
149
+ },
150
+ {
151
+ "epoch": 10.32258064516129,
152
+ "grad_norm": 9.489636421203613,
153
+ "learning_rate": 4.2857142857142856e-05,
154
+ "loss": 0.5332,
155
+ "step": 80
156
+ },
157
+ {
158
+ "epoch": 10.967741935483872,
159
+ "eval_accuracy": 0.7909090909090909,
160
+ "eval_loss": 0.5864469408988953,
161
+ "eval_runtime": 2.4555,
162
+ "eval_samples_per_second": 44.798,
163
+ "eval_steps_per_second": 1.629,
164
+ "step": 85
165
+ },
166
+ {
167
+ "epoch": 11.612903225806452,
168
+ "grad_norm": 6.926851749420166,
169
+ "learning_rate": 4.126984126984127e-05,
170
+ "loss": 0.4483,
171
+ "step": 90
172
+ },
173
+ {
174
+ "epoch": 12.0,
175
+ "eval_accuracy": 0.8272727272727273,
176
+ "eval_loss": 0.49381566047668457,
177
+ "eval_runtime": 2.413,
178
+ "eval_samples_per_second": 45.586,
179
+ "eval_steps_per_second": 1.658,
180
+ "step": 93
181
+ },
182
+ {
183
+ "epoch": 12.903225806451612,
184
+ "grad_norm": 7.890347480773926,
185
+ "learning_rate": 3.968253968253968e-05,
186
+ "loss": 0.378,
187
+ "step": 100
188
+ },
189
+ {
190
+ "epoch": 12.903225806451612,
191
+ "eval_accuracy": 0.8363636363636363,
192
+ "eval_loss": 0.4695802927017212,
193
+ "eval_runtime": 2.3886,
194
+ "eval_samples_per_second": 46.053,
195
+ "eval_steps_per_second": 1.675,
196
+ "step": 100
197
+ },
198
+ {
199
+ "epoch": 13.935483870967742,
200
+ "eval_accuracy": 0.8545454545454545,
201
+ "eval_loss": 0.4418897032737732,
202
+ "eval_runtime": 2.8373,
203
+ "eval_samples_per_second": 38.769,
204
+ "eval_steps_per_second": 1.41,
205
+ "step": 108
206
+ },
207
+ {
208
+ "epoch": 14.193548387096774,
209
+ "grad_norm": 6.673839092254639,
210
+ "learning_rate": 3.809523809523809e-05,
211
+ "loss": 0.3461,
212
+ "step": 110
213
+ },
214
+ {
215
+ "epoch": 14.967741935483872,
216
+ "eval_accuracy": 0.8636363636363636,
217
+ "eval_loss": 0.434987872838974,
218
+ "eval_runtime": 2.4785,
219
+ "eval_samples_per_second": 44.382,
220
+ "eval_steps_per_second": 1.614,
221
+ "step": 116
222
+ },
223
+ {
224
+ "epoch": 15.483870967741936,
225
+ "grad_norm": 6.9846038818359375,
226
+ "learning_rate": 3.650793650793651e-05,
227
+ "loss": 0.333,
228
+ "step": 120
229
+ },
230
+ {
231
+ "epoch": 16.0,
232
+ "eval_accuracy": 0.8727272727272727,
233
+ "eval_loss": 0.42854008078575134,
234
+ "eval_runtime": 2.9037,
235
+ "eval_samples_per_second": 37.882,
236
+ "eval_steps_per_second": 1.378,
237
+ "step": 124
238
+ },
239
+ {
240
+ "epoch": 16.774193548387096,
241
+ "grad_norm": 7.543964385986328,
242
+ "learning_rate": 3.492063492063492e-05,
243
+ "loss": 0.2771,
244
+ "step": 130
245
+ },
246
+ {
247
+ "epoch": 16.903225806451612,
248
+ "eval_accuracy": 0.8636363636363636,
249
+ "eval_loss": 0.41505008935928345,
250
+ "eval_runtime": 2.4017,
251
+ "eval_samples_per_second": 45.801,
252
+ "eval_steps_per_second": 1.665,
253
+ "step": 131
254
+ },
255
+ {
256
+ "epoch": 17.93548387096774,
257
+ "eval_accuracy": 0.8818181818181818,
258
+ "eval_loss": 0.3937516510486603,
259
+ "eval_runtime": 2.4802,
260
+ "eval_samples_per_second": 44.352,
261
+ "eval_steps_per_second": 1.613,
262
+ "step": 139
263
+ },
264
+ {
265
+ "epoch": 18.06451612903226,
266
+ "grad_norm": 7.334432125091553,
267
+ "learning_rate": 3.3333333333333335e-05,
268
+ "loss": 0.2791,
269
+ "step": 140
270
+ },
271
+ {
272
+ "epoch": 18.967741935483872,
273
+ "eval_accuracy": 0.8818181818181818,
274
+ "eval_loss": 0.3853372037410736,
275
+ "eval_runtime": 3.1616,
276
+ "eval_samples_per_second": 34.793,
277
+ "eval_steps_per_second": 1.265,
278
+ "step": 147
279
+ },
280
+ {
281
+ "epoch": 19.35483870967742,
282
+ "grad_norm": 9.302658081054688,
283
+ "learning_rate": 3.1746031746031745e-05,
284
+ "loss": 0.2939,
285
+ "step": 150
286
+ },
287
+ {
288
+ "epoch": 20.0,
289
+ "eval_accuracy": 0.8636363636363636,
290
+ "eval_loss": 0.4061141610145569,
291
+ "eval_runtime": 2.3733,
292
+ "eval_samples_per_second": 46.349,
293
+ "eval_steps_per_second": 1.685,
294
+ "step": 155
295
+ },
296
+ {
297
+ "epoch": 20.64516129032258,
298
+ "grad_norm": 4.140163898468018,
299
+ "learning_rate": 3.0158730158730158e-05,
300
+ "loss": 0.2651,
301
+ "step": 160
302
+ },
303
+ {
304
+ "epoch": 20.903225806451612,
305
+ "eval_accuracy": 0.8545454545454545,
306
+ "eval_loss": 0.44339340925216675,
307
+ "eval_runtime": 2.3755,
308
+ "eval_samples_per_second": 46.306,
309
+ "eval_steps_per_second": 1.684,
310
+ "step": 162
311
+ },
312
+ {
313
+ "epoch": 21.93548387096774,
314
+ "grad_norm": 4.322032928466797,
315
+ "learning_rate": 2.857142857142857e-05,
316
+ "loss": 0.2462,
317
+ "step": 170
318
+ },
319
+ {
320
+ "epoch": 21.93548387096774,
321
+ "eval_accuracy": 0.8909090909090909,
322
+ "eval_loss": 0.3812616765499115,
323
+ "eval_runtime": 2.8623,
324
+ "eval_samples_per_second": 38.431,
325
+ "eval_steps_per_second": 1.397,
326
+ "step": 170
327
+ },
328
+ {
329
+ "epoch": 22.967741935483872,
330
+ "eval_accuracy": 0.8818181818181818,
331
+ "eval_loss": 0.40069419145584106,
332
+ "eval_runtime": 2.3779,
333
+ "eval_samples_per_second": 46.259,
334
+ "eval_steps_per_second": 1.682,
335
+ "step": 178
336
+ },
337
+ {
338
+ "epoch": 23.225806451612904,
339
+ "grad_norm": 5.801928520202637,
340
+ "learning_rate": 2.6984126984126984e-05,
341
+ "loss": 0.2277,
342
+ "step": 180
343
+ },
344
+ {
345
+ "epoch": 24.0,
346
+ "eval_accuracy": 0.8727272727272727,
347
+ "eval_loss": 0.37841716408729553,
348
+ "eval_runtime": 3.0601,
349
+ "eval_samples_per_second": 35.946,
350
+ "eval_steps_per_second": 1.307,
351
+ "step": 186
352
+ },
353
+ {
354
+ "epoch": 24.516129032258064,
355
+ "grad_norm": 6.182375907897949,
356
+ "learning_rate": 2.5396825396825397e-05,
357
+ "loss": 0.2289,
358
+ "step": 190
359
+ },
360
+ {
361
+ "epoch": 24.903225806451612,
362
+ "eval_accuracy": 0.8636363636363636,
363
+ "eval_loss": 0.36822086572647095,
364
+ "eval_runtime": 2.3639,
365
+ "eval_samples_per_second": 46.533,
366
+ "eval_steps_per_second": 1.692,
367
+ "step": 193
368
+ },
369
+ {
370
+ "epoch": 25.806451612903224,
371
+ "grad_norm": 5.690340518951416,
372
+ "learning_rate": 2.380952380952381e-05,
373
+ "loss": 0.2518,
374
+ "step": 200
375
+ },
376
+ {
377
+ "epoch": 25.93548387096774,
378
+ "eval_accuracy": 0.8636363636363636,
379
+ "eval_loss": 0.42345187067985535,
380
+ "eval_runtime": 2.3856,
381
+ "eval_samples_per_second": 46.111,
382
+ "eval_steps_per_second": 1.677,
383
+ "step": 201
384
+ },
385
+ {
386
+ "epoch": 26.967741935483872,
387
+ "eval_accuracy": 0.8727272727272727,
388
+ "eval_loss": 0.4013482928276062,
389
+ "eval_runtime": 3.0149,
390
+ "eval_samples_per_second": 36.485,
391
+ "eval_steps_per_second": 1.327,
392
+ "step": 209
393
+ },
394
+ {
395
+ "epoch": 27.096774193548388,
396
+ "grad_norm": 4.664611339569092,
397
+ "learning_rate": 2.2222222222222223e-05,
398
+ "loss": 0.1961,
399
+ "step": 210
400
+ },
401
+ {
402
+ "epoch": 28.0,
403
+ "eval_accuracy": 0.8727272727272727,
404
+ "eval_loss": 0.37050673365592957,
405
+ "eval_runtime": 2.3916,
406
+ "eval_samples_per_second": 45.995,
407
+ "eval_steps_per_second": 1.673,
408
+ "step": 217
409
+ },
410
+ {
411
+ "epoch": 28.387096774193548,
412
+ "grad_norm": 3.5286619663238525,
413
+ "learning_rate": 2.0634920634920636e-05,
414
+ "loss": 0.2316,
415
+ "step": 220
416
+ },
417
+ {
418
+ "epoch": 28.903225806451612,
419
+ "eval_accuracy": 0.8727272727272727,
420
+ "eval_loss": 0.3901243507862091,
421
+ "eval_runtime": 2.8269,
422
+ "eval_samples_per_second": 38.911,
423
+ "eval_steps_per_second": 1.415,
424
+ "step": 224
425
+ },
426
+ {
427
+ "epoch": 29.677419354838708,
428
+ "grad_norm": 3.580972194671631,
429
+ "learning_rate": 1.9047619047619046e-05,
430
+ "loss": 0.1802,
431
+ "step": 230
432
+ },
433
+ {
434
+ "epoch": 29.93548387096774,
435
+ "eval_accuracy": 0.8636363636363636,
436
+ "eval_loss": 0.4016783535480499,
437
+ "eval_runtime": 2.4102,
438
+ "eval_samples_per_second": 45.64,
439
+ "eval_steps_per_second": 1.66,
440
+ "step": 232
441
+ },
442
+ {
443
+ "epoch": 30.967741935483872,
444
+ "grad_norm": 5.339540481567383,
445
+ "learning_rate": 1.746031746031746e-05,
446
+ "loss": 0.1711,
447
+ "step": 240
448
+ },
449
+ {
450
+ "epoch": 30.967741935483872,
451
+ "eval_accuracy": 0.8454545454545455,
452
+ "eval_loss": 0.40803027153015137,
453
+ "eval_runtime": 2.3509,
454
+ "eval_samples_per_second": 46.791,
455
+ "eval_steps_per_second": 1.701,
456
+ "step": 240
457
+ },
458
+ {
459
+ "epoch": 32.0,
460
+ "eval_accuracy": 0.8636363636363636,
461
+ "eval_loss": 0.37732693552970886,
462
+ "eval_runtime": 3.1157,
463
+ "eval_samples_per_second": 35.305,
464
+ "eval_steps_per_second": 1.284,
465
+ "step": 248
466
+ },
467
+ {
468
+ "epoch": 32.25806451612903,
469
+ "grad_norm": 5.180632591247559,
470
+ "learning_rate": 1.5873015873015872e-05,
471
+ "loss": 0.1885,
472
+ "step": 250
473
+ },
474
+ {
475
+ "epoch": 32.903225806451616,
476
+ "eval_accuracy": 0.8727272727272727,
477
+ "eval_loss": 0.3668518662452698,
478
+ "eval_runtime": 2.3934,
479
+ "eval_samples_per_second": 45.959,
480
+ "eval_steps_per_second": 1.671,
481
+ "step": 255
482
+ },
483
+ {
484
+ "epoch": 33.54838709677419,
485
+ "grad_norm": 4.732351779937744,
486
+ "learning_rate": 1.4285714285714285e-05,
487
+ "loss": 0.1784,
488
+ "step": 260
489
+ },
490
+ {
491
+ "epoch": 33.935483870967744,
492
+ "eval_accuracy": 0.8636363636363636,
493
+ "eval_loss": 0.4083550274372101,
494
+ "eval_runtime": 2.7042,
495
+ "eval_samples_per_second": 40.677,
496
+ "eval_steps_per_second": 1.479,
497
+ "step": 263
498
+ },
499
+ {
500
+ "epoch": 34.83870967741935,
501
+ "grad_norm": 4.103294372558594,
502
+ "learning_rate": 1.2698412698412699e-05,
503
+ "loss": 0.18,
504
+ "step": 270
505
+ },
506
+ {
507
+ "epoch": 34.96774193548387,
508
+ "eval_accuracy": 0.8636363636363636,
509
+ "eval_loss": 0.42064914107322693,
510
+ "eval_runtime": 2.3974,
511
+ "eval_samples_per_second": 45.884,
512
+ "eval_steps_per_second": 1.668,
513
+ "step": 271
514
+ },
515
+ {
516
+ "epoch": 36.0,
517
+ "eval_accuracy": 0.8636363636363636,
518
+ "eval_loss": 0.4106072187423706,
519
+ "eval_runtime": 2.3253,
520
+ "eval_samples_per_second": 47.306,
521
+ "eval_steps_per_second": 1.72,
522
+ "step": 279
523
+ },
524
+ {
525
+ "epoch": 36.12903225806452,
526
+ "grad_norm": 5.152451515197754,
527
+ "learning_rate": 1.1111111111111112e-05,
528
+ "loss": 0.1752,
529
+ "step": 280
530
+ },
531
+ {
532
+ "epoch": 36.903225806451616,
533
+ "eval_accuracy": 0.8727272727272727,
534
+ "eval_loss": 0.41326773166656494,
535
+ "eval_runtime": 3.0981,
536
+ "eval_samples_per_second": 35.506,
537
+ "eval_steps_per_second": 1.291,
538
+ "step": 286
539
+ },
540
+ {
541
+ "epoch": 37.41935483870968,
542
+ "grad_norm": 5.882301330566406,
543
+ "learning_rate": 9.523809523809523e-06,
544
+ "loss": 0.1778,
545
+ "step": 290
546
+ },
547
+ {
548
+ "epoch": 37.935483870967744,
549
+ "eval_accuracy": 0.8727272727272727,
550
+ "eval_loss": 0.41836193203926086,
551
+ "eval_runtime": 2.3256,
552
+ "eval_samples_per_second": 47.299,
553
+ "eval_steps_per_second": 1.72,
554
+ "step": 294
555
+ },
556
+ {
557
+ "epoch": 38.70967741935484,
558
+ "grad_norm": 6.89911413192749,
559
+ "learning_rate": 7.936507936507936e-06,
560
+ "loss": 0.1633,
561
+ "step": 300
562
+ },
563
+ {
564
+ "epoch": 38.96774193548387,
565
+ "eval_accuracy": 0.8636363636363636,
566
+ "eval_loss": 0.4235934615135193,
567
+ "eval_runtime": 2.375,
568
+ "eval_samples_per_second": 46.316,
569
+ "eval_steps_per_second": 1.684,
570
+ "step": 302
571
+ },
572
+ {
573
+ "epoch": 40.0,
574
+ "grad_norm": 8.095017433166504,
575
+ "learning_rate": 6.349206349206349e-06,
576
+ "loss": 0.1621,
577
+ "step": 310
578
+ },
579
+ {
580
+ "epoch": 40.0,
581
+ "eval_accuracy": 0.8727272727272727,
582
+ "eval_loss": 0.4168175756931305,
583
+ "eval_runtime": 2.4838,
584
+ "eval_samples_per_second": 44.287,
585
+ "eval_steps_per_second": 1.61,
586
+ "step": 310
587
+ },
588
+ {
589
+ "epoch": 40.903225806451616,
590
+ "eval_accuracy": 0.8727272727272727,
591
+ "eval_loss": 0.4186895787715912,
592
+ "eval_runtime": 2.3335,
593
+ "eval_samples_per_second": 47.14,
594
+ "eval_steps_per_second": 1.714,
595
+ "step": 317
596
+ },
597
+ {
598
+ "epoch": 41.29032258064516,
599
+ "grad_norm": 9.642603874206543,
600
+ "learning_rate": 4.7619047619047615e-06,
601
+ "loss": 0.1497,
602
+ "step": 320
603
+ },
604
+ {
605
+ "epoch": 41.935483870967744,
606
+ "eval_accuracy": 0.8727272727272727,
607
+ "eval_loss": 0.4140172302722931,
608
+ "eval_runtime": 3.1187,
609
+ "eval_samples_per_second": 35.271,
610
+ "eval_steps_per_second": 1.283,
611
+ "step": 325
612
+ },
613
+ {
614
+ "epoch": 42.58064516129032,
615
+ "grad_norm": 4.230076789855957,
616
+ "learning_rate": 3.1746031746031746e-06,
617
+ "loss": 0.1434,
618
+ "step": 330
619
+ },
620
+ {
621
+ "epoch": 42.96774193548387,
622
+ "eval_accuracy": 0.8909090909090909,
623
+ "eval_loss": 0.41179314255714417,
624
+ "eval_runtime": 2.3523,
625
+ "eval_samples_per_second": 46.763,
626
+ "eval_steps_per_second": 1.7,
627
+ "step": 333
628
+ },
629
+ {
630
+ "epoch": 43.87096774193548,
631
+ "grad_norm": 6.496473789215088,
632
+ "learning_rate": 1.5873015873015873e-06,
633
+ "loss": 0.1802,
634
+ "step": 340
635
+ },
636
+ {
637
+ "epoch": 44.0,
638
+ "eval_accuracy": 0.8818181818181818,
639
+ "eval_loss": 0.412504106760025,
640
+ "eval_runtime": 2.5036,
641
+ "eval_samples_per_second": 43.937,
642
+ "eval_steps_per_second": 1.598,
643
+ "step": 341
644
+ },
645
+ {
646
+ "epoch": 44.903225806451616,
647
+ "eval_accuracy": 0.8727272727272727,
648
+ "eval_loss": 0.4124238193035126,
649
+ "eval_runtime": 2.3607,
650
+ "eval_samples_per_second": 46.596,
651
+ "eval_steps_per_second": 1.694,
652
+ "step": 348
653
+ },
654
+ {
655
+ "epoch": 45.16129032258065,
656
+ "grad_norm": 3.4616761207580566,
657
+ "learning_rate": 0.0,
658
+ "loss": 0.1576,
659
+ "step": 350
660
+ },
661
+ {
662
+ "epoch": 45.16129032258065,
663
+ "eval_accuracy": 0.8727272727272727,
664
+ "eval_loss": 0.4122399687767029,
665
+ "eval_runtime": 3.667,
666
+ "eval_samples_per_second": 29.997,
667
+ "eval_steps_per_second": 1.091,
668
+ "step": 350
669
+ },
670
+ {
671
+ "epoch": 45.16129032258065,
672
+ "step": 350,
673
+ "total_flos": 1.1015215348522291e+18,
674
+ "train_loss": 0.49926152706146243,
675
+ "train_runtime": 2076.2527,
676
+ "train_samples_per_second": 23.624,
677
+ "train_steps_per_second": 0.169
678
+ }
679
+ ],
680
+ "logging_steps": 10,
681
+ "max_steps": 350,
682
+ "num_input_tokens_seen": 0,
683
+ "num_train_epochs": 50,
684
+ "save_steps": 500,
685
+ "stateful_callbacks": {
686
+ "TrainerControl": {
687
+ "args": {
688
+ "should_epoch_stop": false,
689
+ "should_evaluate": false,
690
+ "should_log": false,
691
+ "should_save": true,
692
+ "should_training_stop": true
693
+ },
694
+ "attributes": {}
695
+ }
696
+ },
697
+ "total_flos": 1.1015215348522291e+18,
698
+ "train_batch_size": 32,
699
+ "trial_name": null,
700
+ "trial_params": null
701
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bdb175a7821bbf3b38b17d55bddecd7ca85e70524b057dfe72f177c71f12839a
3
+ size 5240