--- library_name: peft tags: - generated_from_trainer base_model: TheBloke/Llama-2-7B-fp16 model-index: - name: Saiga_timelist_task200steps results: [] --- # Saiga_timelist_task200steps This model is a fine-tuned version of [TheBloke/Llama-2-7B-fp16](https://huggingface.co/TheBloke/Llama-2-7B-fp16) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.4521 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 2 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 10 - total_train_batch_size: 20 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 200 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 2.2298 | 0.37 | 2 | 2.2020 | | 2.0975 | 0.74 | 4 | 2.1478 | | 2.0243 | 1.11 | 6 | 2.1123 | | 1.988 | 1.48 | 8 | 2.0857 | | 1.9585 | 1.85 | 10 | 2.0692 | | 1.883 | 2.22 | 12 | 2.0570 | | 1.9078 | 2.59 | 14 | 2.0477 | | 1.9179 | 2.96 | 16 | 2.0408 | | 1.8663 | 3.33 | 18 | 2.0366 | | 1.8191 | 3.7 | 20 | 2.0325 | | 1.8515 | 4.07 | 22 | 2.0280 | | 1.8189 | 4.44 | 24 | 2.0246 | | 1.8478 | 4.81 | 26 | 2.0215 | | 1.7767 | 5.19 | 28 | 2.0198 | | 1.7685 | 5.56 | 30 | 2.0190 | | 1.7895 | 5.93 | 32 | 2.0189 | | 1.7285 | 6.3 | 34 | 2.0191 | | 1.7609 | 6.67 | 36 | 2.0174 | | 1.7138 | 7.04 | 38 | 2.0156 | | 1.7112 | 7.41 | 40 | 2.0187 | | 1.7029 | 7.78 | 42 | 2.0216 | | 1.6787 | 8.15 | 44 | 2.0203 | | 1.646 | 8.52 | 46 | 2.0243 | | 1.5996 | 8.89 | 48 | 2.0294 | | 1.6838 | 9.26 | 50 | 2.0280 | | 1.6057 | 9.63 | 52 | 2.0254 | | 1.574 | 10.0 | 54 | 2.0310 | | 1.51 | 10.37 | 56 | 2.0547 | | 1.5951 | 10.74 | 58 | 2.0420 | | 1.5455 | 11.11 | 60 | 2.0350 | | 1.5424 | 11.48 | 62 | 2.0612 | | 1.4933 | 11.85 | 64 | 2.0652 | | 1.5766 | 12.22 | 66 | 2.0537 | | 1.4453 | 12.59 | 68 | 2.0732 | | 1.4683 | 12.96 | 70 | 2.0763 | | 1.4734 | 13.33 | 72 | 2.0805 | | 1.4314 | 13.7 | 74 | 2.0908 | | 1.3921 | 14.07 | 76 | 2.0815 | | 1.4099 | 14.44 | 78 | 2.1134 | | 1.4389 | 14.81 | 80 | 2.0955 | | 1.3114 | 15.19 | 82 | 2.1153 | | 1.3093 | 15.56 | 84 | 2.1303 | | 1.3984 | 15.93 | 86 | 2.1246 | | 1.2831 | 16.3 | 88 | 2.1564 | | 1.2971 | 16.67 | 90 | 2.1284 | | 1.3052 | 17.04 | 92 | 2.1608 | | 1.2421 | 17.41 | 94 | 2.1556 | | 1.1835 | 17.78 | 96 | 2.1734 | | 1.283 | 18.15 | 98 | 2.1773 | | 1.2311 | 18.52 | 100 | 2.1992 | | 1.2428 | 18.89 | 102 | 2.1954 | | 1.1959 | 19.26 | 104 | 2.2065 | | 1.2376 | 19.63 | 106 | 2.2124 | | 1.0689 | 20.0 | 108 | 2.2266 | | 1.1471 | 20.37 | 110 | 2.2266 | | 1.0068 | 20.74 | 112 | 2.2451 | | 1.161 | 21.11 | 114 | 2.2501 | | 1.1252 | 21.48 | 116 | 2.2579 | | 1.0683 | 21.85 | 118 | 2.2595 | | 1.1279 | 22.22 | 120 | 2.2904 | | 0.9923 | 22.59 | 122 | 2.2693 | | 1.0139 | 22.96 | 124 | 2.3008 | | 0.9924 | 23.33 | 126 | 2.3036 | | 1.0418 | 23.7 | 128 | 2.3277 | | 1.0463 | 24.07 | 130 | 2.3043 | | 1.0556 | 24.44 | 132 | 2.3262 | | 0.9991 | 24.81 | 134 | 2.3299 | | 0.96 | 25.19 | 136 | 2.3481 | | 0.9677 | 25.56 | 138 | 2.3458 | | 0.9107 | 25.93 | 140 | 2.3607 | | 0.8962 | 26.3 | 142 | 2.3644 | | 0.916 | 26.67 | 144 | 2.3700 | | 0.9284 | 27.04 | 146 | 2.3726 | | 0.99 | 27.41 | 148 | 2.3860 | | 0.8308 | 27.78 | 150 | 2.3918 | | 0.9459 | 28.15 | 152 | 2.3971 | | 0.9283 | 28.52 | 154 | 2.4030 | | 0.863 | 28.89 | 156 | 2.4024 | | 0.9068 | 29.26 | 158 | 2.4083 | | 0.8623 | 29.63 | 160 | 2.4179 | | 0.8359 | 30.0 | 162 | 2.4262 | | 0.953 | 30.37 | 164 | 2.4281 | | 0.7937 | 30.74 | 166 | 2.4381 | | 0.8274 | 31.11 | 168 | 2.4255 | | 0.8862 | 31.48 | 170 | 2.4330 | | 0.7913 | 31.85 | 172 | 2.4511 | | 0.8436 | 32.22 | 174 | 2.4522 | | 0.8519 | 32.59 | 176 | 2.4413 | | 0.8089 | 32.96 | 178 | 2.4371 | | 0.8876 | 33.33 | 180 | 2.4434 | | 0.7836 | 33.7 | 182 | 2.4532 | | 0.8232 | 34.07 | 184 | 2.4566 | | 0.8299 | 34.44 | 186 | 2.4582 | | 0.7977 | 34.81 | 188 | 2.4553 | | 0.8635 | 35.19 | 190 | 2.4522 | | 0.883 | 35.56 | 192 | 2.4518 | | 0.8158 | 35.93 | 194 | 2.4513 | | 0.8732 | 36.3 | 196 | 2.4518 | | 0.8112 | 36.67 | 198 | 2.4522 | | 0.7869 | 37.04 | 200 | 2.4521 | ### Framework versions - PEFT 0.10.0 - Transformers 4.39.3 - Pytorch 2.2.2+cu121 - Datasets 2.18.0 - Tokenizers 0.15.2