Model save
Browse files
README.md
CHANGED
|
@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 18 |
|
| 19 |
This model is a fine-tuned version of [google/codegemma-7b](https://huggingface.co/google/codegemma-7b) on the None dataset.
|
| 20 |
It achieves the following results on the evaluation set:
|
| 21 |
-
- Loss: 0.
|
| 22 |
|
| 23 |
## Model description
|
| 24 |
|
|
@@ -37,7 +37,7 @@ More information needed
|
|
| 37 |
### Training hyperparameters
|
| 38 |
|
| 39 |
The following hyperparameters were used during training:
|
| 40 |
-
- learning_rate:
|
| 41 |
- train_batch_size: 1
|
| 42 |
- eval_batch_size: 3
|
| 43 |
- seed: 42
|
|
@@ -45,125 +45,146 @@ The following hyperparameters were used during training:
|
|
| 45 |
- gradient_accumulation_steps: 8
|
| 46 |
- total_train_batch_size: 8
|
| 47 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 48 |
-
- lr_scheduler_type:
|
| 49 |
- lr_scheduler_warmup_ratio: 0.03
|
| 50 |
-
- num_epochs:
|
| 51 |
- mixed_precision_training: Native AMP
|
| 52 |
|
| 53 |
### Training results
|
| 54 |
|
| 55 |
| Training Loss | Epoch | Step | Validation Loss |
|
| 56 |
|:-------------:|:------:|:----:|:---------------:|
|
| 57 |
-
|
|
| 58 |
-
| 0.
|
| 59 |
-
| 0.
|
| 60 |
-
| 0.
|
| 61 |
-
| 0.
|
| 62 |
-
| 0.
|
| 63 |
-
| 0.
|
| 64 |
-
| 0.
|
| 65 |
-
| 0.
|
| 66 |
-
| 0.
|
| 67 |
-
| 0.
|
| 68 |
-
| 0.
|
| 69 |
-
| 0.
|
| 70 |
-
| 0.
|
| 71 |
-
| 0.
|
| 72 |
-
| 0.
|
| 73 |
-
| 0.
|
| 74 |
-
| 0.
|
| 75 |
-
| 0.
|
| 76 |
-
| 0.
|
| 77 |
-
| 0.
|
| 78 |
-
| 0.
|
| 79 |
-
| 0.
|
| 80 |
-
| 0.
|
| 81 |
-
| 0.
|
| 82 |
-
| 0.
|
| 83 |
-
| 0.
|
| 84 |
-
| 0.
|
| 85 |
-
| 0.
|
| 86 |
-
| 0.
|
| 87 |
-
| 0.
|
| 88 |
-
| 0.
|
| 89 |
-
| 0.
|
| 90 |
-
| 0.
|
| 91 |
-
| 0.
|
| 92 |
-
| 0.
|
| 93 |
-
| 0.
|
| 94 |
-
| 0.
|
| 95 |
-
| 0.
|
| 96 |
-
| 0.
|
| 97 |
-
| 0.
|
| 98 |
-
| 0.
|
| 99 |
-
| 0.
|
| 100 |
-
| 0.
|
| 101 |
-
| 0.
|
| 102 |
-
| 0.
|
| 103 |
-
| 0.
|
| 104 |
-
| 0.
|
| 105 |
-
| 0.
|
| 106 |
-
| 0.
|
| 107 |
-
| 0.
|
| 108 |
-
| 0.
|
| 109 |
-
| 0.
|
| 110 |
-
| 0.
|
| 111 |
-
| 0.
|
| 112 |
-
| 0.
|
| 113 |
-
| 0.
|
| 114 |
-
| 0.
|
| 115 |
-
| 0.
|
| 116 |
-
| 0.
|
| 117 |
-
| 0.
|
| 118 |
-
| 0.
|
| 119 |
-
| 0.
|
| 120 |
-
| 0.
|
| 121 |
-
| 0.
|
| 122 |
-
| 0.
|
| 123 |
-
| 0.
|
| 124 |
-
| 0.
|
| 125 |
-
| 0.
|
| 126 |
-
| 0.
|
| 127 |
-
| 0.
|
| 128 |
-
| 0.
|
| 129 |
-
| 0.
|
| 130 |
-
| 0.
|
| 131 |
-
| 0.
|
| 132 |
-
| 0.
|
| 133 |
-
| 0.
|
| 134 |
-
| 0.
|
| 135 |
-
| 0.
|
| 136 |
-
| 0.
|
| 137 |
-
| 0.
|
| 138 |
-
| 0.
|
| 139 |
-
| 0.
|
| 140 |
-
| 0.
|
| 141 |
-
| 0.
|
| 142 |
-
| 0.
|
| 143 |
-
| 0.
|
| 144 |
-
| 0.
|
| 145 |
-
| 0.
|
| 146 |
-
| 0.
|
| 147 |
-
| 0.
|
| 148 |
-
| 0.
|
| 149 |
-
| 0.
|
| 150 |
-
| 0.
|
| 151 |
-
| 0.
|
| 152 |
-
| 0.
|
| 153 |
-
| 0.
|
| 154 |
-
| 0.
|
| 155 |
-
| 0.
|
| 156 |
-
| 0.
|
| 157 |
-
| 0.
|
| 158 |
-
| 0.
|
| 159 |
-
| 0.
|
| 160 |
-
| 0.
|
| 161 |
-
| 0.
|
| 162 |
-
| 0.0577 | 5.
|
| 163 |
-
| 0.
|
| 164 |
-
| 0.
|
| 165 |
-
| 0.
|
| 166 |
-
| 0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
|
| 168 |
|
| 169 |
### Framework versions
|
|
|
|
| 18 |
|
| 19 |
This model is a fine-tuned version of [google/codegemma-7b](https://huggingface.co/google/codegemma-7b) on the None dataset.
|
| 20 |
It achieves the following results on the evaluation set:
|
| 21 |
+
- Loss: 0.0553
|
| 22 |
|
| 23 |
## Model description
|
| 24 |
|
|
|
|
| 37 |
### Training hyperparameters
|
| 38 |
|
| 39 |
The following hyperparameters were used during training:
|
| 40 |
+
- learning_rate: 5e-05
|
| 41 |
- train_batch_size: 1
|
| 42 |
- eval_batch_size: 3
|
| 43 |
- seed: 42
|
|
|
|
| 45 |
- gradient_accumulation_steps: 8
|
| 46 |
- total_train_batch_size: 8
|
| 47 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 48 |
+
- lr_scheduler_type: cosine
|
| 49 |
- lr_scheduler_warmup_ratio: 0.03
|
| 50 |
+
- num_epochs: 7
|
| 51 |
- mixed_precision_training: Native AMP
|
| 52 |
|
| 53 |
### Training results
|
| 54 |
|
| 55 |
| Training Loss | Epoch | Step | Validation Loss |
|
| 56 |
|:-------------:|:------:|:----:|:---------------:|
|
| 57 |
+
| 2.2877 | 0.0530 | 50 | 1.7325 |
|
| 58 |
+
| 0.7107 | 0.1061 | 100 | 0.6972 |
|
| 59 |
+
| 0.5874 | 0.1591 | 150 | 0.5366 |
|
| 60 |
+
| 0.4688 | 0.2121 | 200 | 0.4286 |
|
| 61 |
+
| 0.386 | 0.2652 | 250 | 0.3401 |
|
| 62 |
+
| 0.2728 | 0.3182 | 300 | 0.2616 |
|
| 63 |
+
| 0.2257 | 0.3713 | 350 | 0.2191 |
|
| 64 |
+
| 0.1962 | 0.4243 | 400 | 0.1729 |
|
| 65 |
+
| 0.1726 | 0.4773 | 450 | 0.1531 |
|
| 66 |
+
| 0.1569 | 0.5304 | 500 | 0.1439 |
|
| 67 |
+
| 0.186 | 0.5834 | 550 | 0.1374 |
|
| 68 |
+
| 0.1467 | 0.6364 | 600 | 0.1326 |
|
| 69 |
+
| 0.1496 | 0.6895 | 650 | 0.1285 |
|
| 70 |
+
| 0.1484 | 0.7425 | 700 | 0.1265 |
|
| 71 |
+
| 0.1345 | 0.7955 | 750 | 0.1232 |
|
| 72 |
+
| 0.1321 | 0.8486 | 800 | 0.1199 |
|
| 73 |
+
| 0.1459 | 0.9016 | 850 | 0.1203 |
|
| 74 |
+
| 0.1305 | 0.9547 | 900 | 0.1174 |
|
| 75 |
+
| 0.1185 | 1.0077 | 950 | 0.1120 |
|
| 76 |
+
| 0.1208 | 1.0607 | 1000 | 0.1110 |
|
| 77 |
+
| 0.1112 | 1.1138 | 1050 | 0.1094 |
|
| 78 |
+
| 0.1154 | 1.1668 | 1100 | 0.1079 |
|
| 79 |
+
| 0.1131 | 1.2198 | 1150 | 0.1071 |
|
| 80 |
+
| 0.1192 | 1.2729 | 1200 | 0.1066 |
|
| 81 |
+
| 0.1217 | 1.3259 | 1250 | 0.1058 |
|
| 82 |
+
| 0.1055 | 1.3789 | 1300 | 0.1055 |
|
| 83 |
+
| 0.1032 | 1.4320 | 1350 | 0.1032 |
|
| 84 |
+
| 0.1133 | 1.4850 | 1400 | 0.1034 |
|
| 85 |
+
| 0.1142 | 1.5381 | 1450 | 0.1022 |
|
| 86 |
+
| 0.1091 | 1.5911 | 1500 | 0.1020 |
|
| 87 |
+
| 0.1024 | 1.6441 | 1550 | 0.0995 |
|
| 88 |
+
| 0.1166 | 1.6972 | 1600 | 0.1002 |
|
| 89 |
+
| 0.1057 | 1.7502 | 1650 | 0.0994 |
|
| 90 |
+
| 0.1056 | 1.8032 | 1700 | 0.0971 |
|
| 91 |
+
| 0.1059 | 1.8563 | 1750 | 0.0972 |
|
| 92 |
+
| 0.1063 | 1.9093 | 1800 | 0.0959 |
|
| 93 |
+
| 0.1025 | 1.9623 | 1850 | 0.0949 |
|
| 94 |
+
| 0.0938 | 2.0154 | 1900 | 0.0949 |
|
| 95 |
+
| 0.0973 | 2.0684 | 1950 | 0.0941 |
|
| 96 |
+
| 0.1037 | 2.1215 | 2000 | 0.0933 |
|
| 97 |
+
| 0.0888 | 2.1745 | 2050 | 0.0924 |
|
| 98 |
+
| 0.0877 | 2.2275 | 2100 | 0.0917 |
|
| 99 |
+
| 0.0874 | 2.2806 | 2150 | 0.0918 |
|
| 100 |
+
| 0.0874 | 2.3336 | 2200 | 0.0907 |
|
| 101 |
+
| 0.0959 | 2.3866 | 2250 | 0.0898 |
|
| 102 |
+
| 0.0954 | 2.4397 | 2300 | 0.0904 |
|
| 103 |
+
| 0.0887 | 2.4927 | 2350 | 0.0885 |
|
| 104 |
+
| 0.0827 | 2.5457 | 2400 | 0.0886 |
|
| 105 |
+
| 0.086 | 2.5988 | 2450 | 0.0869 |
|
| 106 |
+
| 0.0896 | 2.6518 | 2500 | 0.0861 |
|
| 107 |
+
| 0.0875 | 2.7049 | 2550 | 0.0862 |
|
| 108 |
+
| 0.0872 | 2.7579 | 2600 | 0.0863 |
|
| 109 |
+
| 0.0871 | 2.8109 | 2650 | 0.0850 |
|
| 110 |
+
| 0.0901 | 2.8640 | 2700 | 0.0842 |
|
| 111 |
+
| 0.078 | 2.9170 | 2750 | 0.0837 |
|
| 112 |
+
| 0.0878 | 2.9700 | 2800 | 0.0833 |
|
| 113 |
+
| 0.0807 | 3.0231 | 2850 | 0.0828 |
|
| 114 |
+
| 0.0846 | 3.0761 | 2900 | 0.0827 |
|
| 115 |
+
| 0.0741 | 3.1291 | 2950 | 0.0824 |
|
| 116 |
+
| 0.0778 | 3.1822 | 3000 | 0.0821 |
|
| 117 |
+
| 0.0755 | 3.2352 | 3050 | 0.0822 |
|
| 118 |
+
| 0.0728 | 3.2883 | 3100 | 0.0812 |
|
| 119 |
+
| 0.0758 | 3.3413 | 3150 | 0.0816 |
|
| 120 |
+
| 0.0779 | 3.3943 | 3200 | 0.0796 |
|
| 121 |
+
| 0.0705 | 3.4474 | 3250 | 0.0788 |
|
| 122 |
+
| 0.0725 | 3.5004 | 3300 | 0.0783 |
|
| 123 |
+
| 0.0707 | 3.5534 | 3350 | 0.0787 |
|
| 124 |
+
| 0.0659 | 3.6065 | 3400 | 0.0783 |
|
| 125 |
+
| 0.0698 | 3.6595 | 3450 | 0.0780 |
|
| 126 |
+
| 0.0702 | 3.7125 | 3500 | 0.0766 |
|
| 127 |
+
| 0.07 | 3.7656 | 3550 | 0.0768 |
|
| 128 |
+
| 0.0673 | 3.8186 | 3600 | 0.0760 |
|
| 129 |
+
| 0.0706 | 3.8717 | 3650 | 0.0751 |
|
| 130 |
+
| 0.0633 | 3.9247 | 3700 | 0.0741 |
|
| 131 |
+
| 0.0766 | 3.9777 | 3750 | 0.0740 |
|
| 132 |
+
| 0.0597 | 4.0308 | 3800 | 0.0741 |
|
| 133 |
+
| 0.0541 | 4.0838 | 3850 | 0.0742 |
|
| 134 |
+
| 0.0574 | 4.1368 | 3900 | 0.0734 |
|
| 135 |
+
| 0.0611 | 4.1899 | 3950 | 0.0727 |
|
| 136 |
+
| 0.0651 | 4.2429 | 4000 | 0.0723 |
|
| 137 |
+
| 0.0766 | 4.2991 | 4050 | 0.0699 |
|
| 138 |
+
| 0.0736 | 4.3522 | 4100 | 0.0696 |
|
| 139 |
+
| 0.0755 | 4.4052 | 4150 | 0.0695 |
|
| 140 |
+
| 0.0736 | 4.4582 | 4200 | 0.0692 |
|
| 141 |
+
| 0.0721 | 4.5113 | 4250 | 0.0686 |
|
| 142 |
+
| 0.071 | 4.5643 | 4300 | 0.0680 |
|
| 143 |
+
| 0.0675 | 4.6173 | 4350 | 0.0679 |
|
| 144 |
+
| 0.0714 | 4.6704 | 4400 | 0.0674 |
|
| 145 |
+
| 0.0648 | 4.7234 | 4450 | 0.0669 |
|
| 146 |
+
| 0.0729 | 4.7765 | 4500 | 0.0665 |
|
| 147 |
+
| 0.0656 | 4.8295 | 4550 | 0.0660 |
|
| 148 |
+
| 0.0707 | 4.8825 | 4600 | 0.0659 |
|
| 149 |
+
| 0.0703 | 4.9356 | 4650 | 0.0652 |
|
| 150 |
+
| 0.0669 | 4.9886 | 4700 | 0.0647 |
|
| 151 |
+
| 0.0665 | 5.0416 | 4750 | 0.0643 |
|
| 152 |
+
| 0.0573 | 5.0947 | 4800 | 0.0646 |
|
| 153 |
+
| 0.0628 | 5.1477 | 4850 | 0.0642 |
|
| 154 |
+
| 0.0574 | 5.2007 | 4900 | 0.0637 |
|
| 155 |
+
| 0.067 | 5.2538 | 4950 | 0.0632 |
|
| 156 |
+
| 0.06 | 5.3068 | 5000 | 0.0631 |
|
| 157 |
+
| 0.0637 | 5.3599 | 5050 | 0.0632 |
|
| 158 |
+
| 0.0602 | 5.4129 | 5100 | 0.0623 |
|
| 159 |
+
| 0.0592 | 5.4659 | 5150 | 0.0624 |
|
| 160 |
+
| 0.0567 | 5.5190 | 5200 | 0.0616 |
|
| 161 |
+
| 0.0634 | 5.5720 | 5250 | 0.0615 |
|
| 162 |
+
| 0.0577 | 5.6250 | 5300 | 0.0609 |
|
| 163 |
+
| 0.0534 | 5.6781 | 5350 | 0.0609 |
|
| 164 |
+
| 0.0555 | 5.7311 | 5400 | 0.0613 |
|
| 165 |
+
| 0.0601 | 5.7841 | 5450 | 0.0603 |
|
| 166 |
+
| 0.0533 | 5.8372 | 5500 | 0.0610 |
|
| 167 |
+
| 0.0598 | 5.8902 | 5550 | 0.0598 |
|
| 168 |
+
| 0.0603 | 5.9433 | 5600 | 0.0593 |
|
| 169 |
+
| 0.0606 | 5.9963 | 5650 | 0.0592 |
|
| 170 |
+
| 0.047 | 6.0493 | 5700 | 0.0596 |
|
| 171 |
+
| 0.0468 | 6.1024 | 5750 | 0.0589 |
|
| 172 |
+
| 0.0521 | 6.1554 | 5800 | 0.0586 |
|
| 173 |
+
| 0.045 | 6.2084 | 5850 | 0.0592 |
|
| 174 |
+
| 0.0504 | 6.2615 | 5900 | 0.0581 |
|
| 175 |
+
| 0.0474 | 6.3145 | 5950 | 0.0581 |
|
| 176 |
+
| 0.0497 | 6.3675 | 6000 | 0.0583 |
|
| 177 |
+
| 0.0519 | 6.4206 | 6050 | 0.0579 |
|
| 178 |
+
| 0.0467 | 6.4736 | 6100 | 0.0578 |
|
| 179 |
+
| 0.0475 | 6.5267 | 6150 | 0.0573 |
|
| 180 |
+
| 0.0506 | 6.5797 | 6200 | 0.0569 |
|
| 181 |
+
| 0.0486 | 6.6327 | 6250 | 0.0563 |
|
| 182 |
+
| 0.0441 | 6.6858 | 6300 | 0.0564 |
|
| 183 |
+
| 0.0528 | 6.7388 | 6350 | 0.0560 |
|
| 184 |
+
| 0.0506 | 6.7918 | 6400 | 0.0553 |
|
| 185 |
+
| 0.0491 | 6.8449 | 6450 | 0.0554 |
|
| 186 |
+
| 0.0458 | 6.8979 | 6500 | 0.0550 |
|
| 187 |
+
| 0.0458 | 6.9509 | 6550 | 0.0553 |
|
| 188 |
|
| 189 |
|
| 190 |
### Framework versions
|
adapter_model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4745934024
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f4a19481e8f4debfee25cd001550f23ea827cab4a07a56bed84a8b614f3ff78
|
| 3 |
size 4745934024
|
runs/Sep13_08-34-49_m3h110/events.out.tfevents.1757717579.m3h110.277836.0
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8cac6b06f9ebb2f506b5c2e6ff7941fee8b1c803435caf477dc72fdf54a00836
|
| 3 |
+
size 74222
|