Hugging Face's logo

jkazdan
/
collapse_gemma-2-2b_hs2_accumulate_iter3_sftsd2

Safetensors
gemma2
trl
sft
Generated from Trainer
Model card Files Community
jkazdan commited on
Commit
c00efc8
·
verified ·
1 Parent(s): 46dbf0d

End of training

Browse files
Files changed (4) hide show
  1. README.md +80 -80
  2. model-00001-of-00002.safetensors +1 -1
  3. model-00002-of-00002.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.1032
21
- - Num Input Tokens Seen: 21819352
22
 
23
  ## Model description
24
 
@@ -53,84 +53,84 @@ The following hyperparameters were used during training:
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
- | 1.5432 | 0.0127 | 5 | 1.3798 | 276728 |
57
- | 1.4208 | 0.0254 | 10 | 1.2917 | 554256 |
58
- | 1.4236 | 0.0381 | 15 | 1.2111 | 833616 |
59
- | 1.3033 | 0.0508 | 20 | 1.1647 | 1109744 |
60
- | 1.2167 | 0.0634 | 25 | 1.1518 | 1384696 |
61
- | 1.0953 | 0.0761 | 30 | 1.1341 | 1664008 |
62
- | 0.9168 | 0.0888 | 35 | 1.1461 | 1944176 |
63
- | 0.9273 | 0.1015 | 40 | 1.1542 | 2218368 |
64
- | 0.8943 | 0.1142 | 45 | 1.1696 | 2492552 |
65
- | 0.8168 | 0.1269 | 50 | 1.1792 | 2773488 |
66
- | 0.7781 | 0.1396 | 55 | 1.1739 | 3050208 |
67
- | 0.8131 | 0.1523 | 60 | 1.1845 | 3326584 |
68
- | 0.6973 | 0.1649 | 65 | 1.1836 | 3606104 |
69
- | 0.7054 | 0.1776 | 70 | 1.1733 | 3887952 |
70
- | 0.685 | 0.1903 | 75 | 1.1764 | 4170752 |
71
- | 0.5768 | 0.2030 | 80 | 1.1771 | 4444816 |
72
- | 0.6494 | 0.2157 | 85 | 1.1719 | 4718552 |
73
- | 0.5484 | 0.2284 | 90 | 1.1698 | 4998784 |
74
- | 0.5609 | 0.2411 | 95 | 1.1739 | 5274536 |
75
- | 0.4343 | 0.2538 | 100 | 1.1755 | 5553760 |
76
- | 0.5656 | 0.2665 | 105 | 1.1654 | 5828328 |
77
- | 0.5633 | 0.2791 | 110 | 1.1696 | 6104712 |
78
- | 0.4485 | 0.2918 | 115 | 1.1631 | 6380840 |
79
- | 0.4853 | 0.3045 | 120 | 1.1658 | 6651752 |
80
- | 0.4552 | 0.3172 | 125 | 1.1593 | 6928872 |
81
- | 0.4465 | 0.3299 | 130 | 1.1584 | 7200200 |
82
- | 0.4402 | 0.3426 | 135 | 1.1605 | 7481976 |
83
- | 0.4228 | 0.3553 | 140 | 1.1536 | 7765000 |
84
- | 0.5075 | 0.3680 | 145 | 1.1529 | 8037040 |
85
- | 0.3783 | 0.3807 | 150 | 1.1505 | 8313288 |
86
- | 0.4 | 0.3933 | 155 | 1.1464 | 8593584 |
87
- | 0.4482 | 0.4060 | 160 | 1.1507 | 8869384 |
88
- | 0.4995 | 0.4187 | 165 | 1.1418 | 9145296 |
89
- | 0.4386 | 0.4314 | 170 | 1.1420 | 9423816 |
90
- | 0.3944 | 0.4441 | 175 | 1.1406 | 9707024 |
91
- | 0.5069 | 0.4568 | 180 | 1.1408 | 9977424 |
92
- | 0.36 | 0.4695 | 185 | 1.1408 | 10247568 |
93
- | 0.4558 | 0.4822 | 190 | 1.1369 | 10525312 |
94
- | 0.4699 | 0.4948 | 195 | 1.1341 | 10807080 |
95
- | 0.5118 | 0.5075 | 200 | 1.1346 | 11075200 |
96
- | 0.5246 | 0.5202 | 205 | 1.1310 | 11355128 |
97
- | 0.5085 | 0.5329 | 210 | 1.1323 | 11635976 |
98
- | 0.3497 | 0.5456 | 215 | 1.1290 | 11912608 |
99
- | 0.4282 | 0.5583 | 220 | 1.1304 | 12191360 |
100
- | 0.3405 | 0.5710 | 225 | 1.1261 | 12468896 |
101
- | 0.4814 | 0.5837 | 230 | 1.1271 | 12748408 |
102
- | 0.3857 | 0.5964 | 235 | 1.1262 | 13023016 |
103
- | 0.4579 | 0.6090 | 240 | 1.1245 | 13302328 |
104
- | 0.4054 | 0.6217 | 245 | 1.1244 | 13575408 |
105
- | 0.4019 | 0.6344 | 250 | 1.1222 | 13851880 |
106
- | 0.4085 | 0.6471 | 255 | 1.1206 | 14126456 |
107
- | 0.3261 | 0.6598 | 260 | 1.1226 | 14411880 |
108
- | 0.3434 | 0.6725 | 265 | 1.1197 | 14693704 |
109
- | 0.3898 | 0.6852 | 270 | 1.1189 | 14972552 |
110
- | 0.3275 | 0.6979 | 275 | 1.1202 | 15244856 |
111
- | 0.3851 | 0.7105 | 280 | 1.1181 | 15517984 |
112
- | 0.3896 | 0.7232 | 285 | 1.1167 | 15793480 |
113
- | 0.4382 | 0.7359 | 290 | 1.1164 | 16072136 |
114
- | 0.4112 | 0.7486 | 295 | 1.1147 | 16347632 |
115
- | 0.4165 | 0.7613 | 300 | 1.1153 | 16622200 |
116
- | 0.3549 | 0.7740 | 305 | 1.1137 | 16896656 |
117
- | 0.3859 | 0.7867 | 310 | 1.1130 | 17175712 |
118
- | 0.3636 | 0.7994 | 315 | 1.1129 | 17456320 |
119
- | 0.4647 | 0.8121 | 320 | 1.1109 | 17735952 |
120
- | 0.3973 | 0.8247 | 325 | 1.1121 | 18011048 |
121
- | 0.3857 | 0.8374 | 330 | 1.1100 | 18285984 |
122
- | 0.3692 | 0.8501 | 335 | 1.1105 | 18560024 |
123
- | 0.4178 | 0.8628 | 340 | 1.1092 | 18834584 |
124
- | 0.3232 | 0.8755 | 345 | 1.1070 | 19113832 |
125
- | 0.3482 | 0.8882 | 350 | 1.1070 | 19390200 |
126
- | 0.4256 | 0.9009 | 355 | 1.1065 | 19670664 |
127
- | 0.4421 | 0.9136 | 360 | 1.1040 | 19946664 |
128
- | 0.4513 | 0.9262 | 365 | 1.1046 | 20229584 |
129
- | 0.395 | 0.9389 | 370 | 1.1059 | 20503736 |
130
- | 0.3129 | 0.9516 | 375 | 1.1033 | 20776680 |
131
- | 0.3915 | 0.9643 | 380 | 1.1048 | 21053616 |
132
- | 0.3239 | 0.9770 | 385 | 1.1003 | 21327312 |
133
- | 0.3765 | 0.9897 | 390 | 1.1039 | 21601936 |
134
 
135
 
136
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.1022
21
+ - Num Input Tokens Seen: 22054048
22
 
23
  ## Model description
24
 
 
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
+ | 1.628 | 0.0127 | 5 | 1.3800 | 282000 |
57
+ | 1.6129 | 0.0254 | 10 | 1.2915 | 565768 |
58
+ | 1.4755 | 0.0381 | 15 | 1.2119 | 845776 |
59
+ | 1.2663 | 0.0508 | 20 | 1.1654 | 1119976 |
60
+ | 1.2503 | 0.0636 | 25 | 1.1530 | 1405752 |
61
+ | 1.0375 | 0.0763 | 30 | 1.1358 | 1683544 |
62
+ | 0.9388 | 0.0890 | 35 | 1.1575 | 1962800 |
63
+ | 0.8887 | 0.1017 | 40 | 1.1613 | 2242448 |
64
+ | 0.9444 | 0.1144 | 45 | 1.1814 | 2530200 |
65
+ | 0.8274 | 0.1271 | 50 | 1.1685 | 2813744 |
66
+ | 0.7725 | 0.1398 | 55 | 1.1846 | 3088392 |
67
+ | 0.7435 | 0.1525 | 60 | 1.1750 | 3367968 |
68
+ | 0.8112 | 0.1652 | 65 | 1.1798 | 3653616 |
69
+ | 0.6116 | 0.1779 | 70 | 1.1803 | 3935936 |
70
+ | 0.6364 | 0.1907 | 75 | 1.1648 | 4215056 |
71
+ | 0.6888 | 0.2034 | 80 | 1.1682 | 4498800 |
72
+ | 0.6489 | 0.2161 | 85 | 1.1755 | 4777456 |
73
+ | 0.5009 | 0.2288 | 90 | 1.1711 | 5056576 |
74
+ | 0.6014 | 0.2415 | 95 | 1.1619 | 5333256 |
75
+ | 0.6265 | 0.2542 | 100 | 1.1702 | 5607960 |
76
+ | 0.4422 | 0.2669 | 105 | 1.1616 | 5888544 |
77
+ | 0.5504 | 0.2796 | 110 | 1.1721 | 6157688 |
78
+ | 0.5325 | 0.2923 | 115 | 1.1638 | 6436816 |
79
+ | 0.4722 | 0.3051 | 120 | 1.1622 | 6720832 |
80
+ | 0.3832 | 0.3178 | 125 | 1.1592 | 7010752 |
81
+ | 0.5639 | 0.3305 | 130 | 1.1548 | 7296936 |
82
+ | 0.4615 | 0.3432 | 135 | 1.1555 | 7569880 |
83
+ | 0.5294 | 0.3559 | 140 | 1.1487 | 7848792 |
84
+ | 0.4983 | 0.3686 | 145 | 1.1543 | 8130552 |
85
+ | 0.4877 | 0.3813 | 150 | 1.1442 | 8409680 |
86
+ | 0.419 | 0.3940 | 155 | 1.1497 | 8691616 |
87
+ | 0.5136 | 0.4067 | 160 | 1.1437 | 8974984 |
88
+ | 0.4672 | 0.4194 | 165 | 1.1442 | 9258208 |
89
+ | 0.4665 | 0.4322 | 170 | 1.1359 | 9538392 |
90
+ | 0.4105 | 0.4449 | 175 | 1.1412 | 9818464 |
91
+ | 0.5283 | 0.4576 | 180 | 1.1360 | 10102088 |
92
+ | 0.4097 | 0.4703 | 185 | 1.1388 | 10385664 |
93
+ | 0.4573 | 0.4830 | 190 | 1.1324 | 10667816 |
94
+ | 0.4047 | 0.4957 | 195 | 1.1343 | 10947272 |
95
+ | 0.4657 | 0.5084 | 200 | 1.1281 | 11227664 |
96
+ | 0.3811 | 0.5211 | 205 | 1.1295 | 11509152 |
97
+ | 0.43 | 0.5338 | 210 | 1.1294 | 11792720 |
98
+ | 0.4653 | 0.5466 | 215 | 1.1250 | 12068688 |
99
+ | 0.3614 | 0.5593 | 220 | 1.1273 | 12350648 |
100
+ | 0.4405 | 0.5720 | 225 | 1.1234 | 12628784 |
101
+ | 0.3511 | 0.5847 | 230 | 1.1251 | 12907416 |
102
+ | 0.4004 | 0.5974 | 235 | 1.1223 | 13192632 |
103
+ | 0.4819 | 0.6101 | 240 | 1.1201 | 13469328 |
104
+ | 0.4378 | 0.6228 | 245 | 1.1201 | 13748984 |
105
+ | 0.3615 | 0.6355 | 250 | 1.1166 | 14033560 |
106
+ | 0.3767 | 0.6482 | 255 | 1.1185 | 14315712 |
107
+ | 0.3775 | 0.6609 | 260 | 1.1169 | 14599040 |
108
+ | 0.4632 | 0.6737 | 265 | 1.1152 | 14883880 |
109
+ | 0.3246 | 0.6864 | 270 | 1.1148 | 15161064 |
110
+ | 0.3381 | 0.6991 | 275 | 1.1136 | 15435968 |
111
+ | 0.3762 | 0.7118 | 280 | 1.1167 | 15715000 |
112
+ | 0.3853 | 0.7245 | 285 | 1.1128 | 15992552 |
113
+ | 0.4548 | 0.7372 | 290 | 1.1124 | 16277624 |
114
+ | 0.3692 | 0.7499 | 295 | 1.1102 | 16554696 |
115
+ | 0.423 | 0.7626 | 300 | 1.1101 | 16842640 |
116
+ | 0.3635 | 0.7753 | 305 | 1.1124 | 17126528 |
117
+ | 0.3939 | 0.7881 | 310 | 1.1096 | 17402024 |
118
+ | 0.4323 | 0.8008 | 315 | 1.1092 | 17679664 |
119
+ | 0.3539 | 0.8135 | 320 | 1.1073 | 17959928 |
120
+ | 0.4876 | 0.8262 | 325 | 1.1077 | 18241888 |
121
+ | 0.3201 | 0.8389 | 330 | 1.1077 | 18521608 |
122
+ | 0.3806 | 0.8516 | 335 | 1.1060 | 18805032 |
123
+ | 0.3601 | 0.8643 | 340 | 1.1062 | 19089648 |
124
+ | 0.3919 | 0.8770 | 345 | 1.1049 | 19371096 |
125
+ | 0.3816 | 0.8897 | 350 | 1.1069 | 19650992 |
126
+ | 0.3584 | 0.9024 | 355 | 1.1051 | 19923856 |
127
+ | 0.3534 | 0.9152 | 360 | 1.1057 | 20198240 |
128
+ | 0.4761 | 0.9279 | 365 | 1.1049 | 20480400 |
129
+ | 0.3723 | 0.9406 | 370 | 1.1053 | 20761832 |
130
+ | 0.4056 | 0.9533 | 375 | 1.1036 | 21048104 |
131
+ | 0.3886 | 0.9660 | 380 | 1.1024 | 21323808 |
132
+ | 0.5005 | 0.9787 | 385 | 1.1028 | 21602888 |
133
+ | 0.3638 | 0.9914 | 390 | 1.1039 | 21887576 |
134
 
135
 
136
  ### Framework versions
model-00001-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:75c0784ae7094b1d066727216660ab56fae1d771aba4d5f69be72e93d150ca34
3
  size 4988025760
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:137f44cf7a048c21829e04aaeb571ddd43ab01ea1843275f982f8fada03aa551
3
  size 4988025760
model-00002-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f973b256cf70e055497504477a8bc4b65a3a6853c6306b7d91f34167eb9dc04f
3
  size 240691728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8115c43edcbda58986545ab500121b2631f3a4a781a00941040788e86eef4cc4
3
  size 240691728
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fe430511dc1fbad736fae0e996a63c2dc2dc8b9f50dafa7a757d4533bc706228
3
  size 5560
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5415960a91a5b59a86ab048b9c81d1428b9cd67b37c19bb3a2d6752678c47003
3
  size 5560