Saiga_timelist_task20steps

README.md

@@ -13,9 +13,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # Saiga_timelist_task20steps
 
-This model is a fine-tuned version of [TheBloke/Llama-2-7B-fp16](https://huggingface.co/TheBloke/Llama-2-7B-fp16) on the None dataset.
+This model is a fine-tuned version of [TheBloke/Llama-2-7B-fp16](https://huggingface.co/TheBloke/Llama-2-7B-fp16) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7498
+- Loss: 2.0584
 
 ## Model description
 
@@ -42,28 +42,22 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 20
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- training_steps: 80
+- training_steps: 20
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.9766        | 0.13  | 5    | 1.9730          |
-| 1.8885        | 0.26  | 10   | 1.8875          |
-| 1.7781        | 0.39  | 15   | 1.8522          |
-| 1.7543        | 0.52  | 20   | 1.8314          |
-| 1.716         | 0.64  | 25   | 1.8151          |
-| 1.8085        | 0.77  | 30   | 1.8015          |
-| 1.6707        | 0.9   | 35   | 1.7902          |
-| 1.6972        | 1.03  | 40   | 1.7805          |
-| 1.6439        | 1.16  | 45   | 1.7728          |
-| 1.6487        | 1.29  | 50   | 1.7668          |
-| 1.5462        | 1.42  | 55   | 1.7606          |
-| 1.6728        | 1.55  | 60   | 1.7557          |
-| 1.6285        | 1.68  | 65   | 1.7520          |
-| 1.5609        | 1.8   | 70   | 1.7508          |
-| 1.5975        | 1.93  | 75   | 1.7500          |
-| 1.6035        | 2.06  | 80   | 1.7498          |
+| 2.2298        | 0.37  | 2    | 2.2031          |
+| 2.0996        | 0.74  | 4    | 2.1519          |
+| 2.0299        | 1.11  | 6    | 2.1202          |
+| 2.0007        | 1.48  | 8    | 2.0978          |
+| 1.9777        | 1.85  | 10   | 2.0817          |
+| 1.9089        | 2.22  | 12   | 2.0715          |
+| 1.9379        | 2.59  | 14   | 2.0650          |
+| 1.9515        | 2.96  | 16   | 2.0610          |
+| 1.9178        | 3.33  | 18   | 2.0589          |
+| 1.8801        | 3.7   | 20   | 2.0584          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,10 +20,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
     "v_proj",
+    "q_proj",
     "o_proj",
-    "q_proj"
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cf7d4cee3da167364c19e2438363e31e6dde39fd24e11d317511148cb3b92065
+oid sha256:135395a2cd4784a47a927003e74b0c497f45818b038657dae23bc8150a0e0dd8
 size 33589040

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7c632e1dcb43c6d589f5104dea52976de3830b6248b57286e3b64a3009dba283
+oid sha256:9f15c661378faf1acf87035a24d714e630dc819d4ea7695c7224a5e79ecc783f
 size 4920