Hugging Face's logo

Christine789
/
shawgpt-ft

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
Christine789 commited on
Commit
f27d982
•
1 Parent(s): 6a68c7a

Christine789/QLoRA32

Browse files
Files changed (18) hide show
  1. README.md +23 -14
  2. adapter_config.json +1 -1
  3. adapter_model.safetensors +2 -2
  4. runs/Oct01_07-07-59_800808cfb2f8/events.out.tfevents.1727766479.800808cfb2f8.4374.1 +3 -0
  5. runs/Oct01_07-20-08_800808cfb2f8/events.out.tfevents.1727767209.800808cfb2f8.4374.2 +3 -0
  6. runs/Oct01_07-25-12_800808cfb2f8/events.out.tfevents.1727767513.800808cfb2f8.4374.3 +3 -0
  7. runs/Oct01_07-29-40_800808cfb2f8/events.out.tfevents.1727767780.800808cfb2f8.4374.4 +3 -0
  8. runs/Oct01_07-33-04_800808cfb2f8/events.out.tfevents.1727767984.800808cfb2f8.4374.5 +3 -0
  9. runs/Oct01_07-36-27_800808cfb2f8/events.out.tfevents.1727768188.800808cfb2f8.4374.6 +3 -0
  10. runs/Oct01_07-59-17_800808cfb2f8/events.out.tfevents.1727769557.800808cfb2f8.4374.7 +3 -0
  11. runs/Oct01_08-03-03_800808cfb2f8/events.out.tfevents.1727769783.800808cfb2f8.4374.8 +3 -0
  12. runs/Oct01_08-21-33_800808cfb2f8/events.out.tfevents.1727770894.800808cfb2f8.4374.9 +3 -0
  13. runs/Oct01_08-28-24_800808cfb2f8/events.out.tfevents.1727771308.800808cfb2f8.4374.10 +3 -0
  14. runs/Oct01_08-28-24_800808cfb2f8/events.out.tfevents.1727771323.800808cfb2f8.4374.11 +3 -0
  15. runs/Oct01_08-36-25_800808cfb2f8/events.out.tfevents.1727771788.800808cfb2f8.4374.12 +3 -0
  16. runs/Oct01_08-42-19_800808cfb2f8/events.out.tfevents.1727772139.800808cfb2f8.4374.13 +3 -0
  17. runs/Oct01_08-48-12_800808cfb2f8/events.out.tfevents.1727772492.800808cfb2f8.4374.14 +3 -0
  18. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 1.7356
20
 
21
  ## Model description
22
 
@@ -44,23 +44,32 @@ The following hyperparameters were used during training:
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
- - num_epochs: 10
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
- | Training Loss | Epoch | Step | Validation Loss |
53
- |:-------------:|:------:|:----:|:---------------:|
54
- | 4.5915 | 0.9231 | 3 | 3.9621 |
55
- | 4.04 | 1.8462 | 6 | 3.4349 |
56
- | 3.4599 | 2.7692 | 9 | 2.9746 |
57
- | 2.2493 | 4.0 | 13 | 2.5418 |
58
- | 2.6545 | 4.9231 | 16 | 2.2966 |
59
- | 2.3332 | 5.8462 | 19 | 2.0944 |
60
- | 2.0967 | 6.7692 | 22 | 1.9367 |
61
- | 1.4509 | 8.0 | 26 | 1.7868 |
62
- | 1.8264 | 8.9231 | 29 | 1.7411 |
63
- | 1.2705 | 9.2308 | 30 | 1.7356 |
 
 
 
 
 
 
 
 
 
64
 
65
 
66
  ### Framework versions
 
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.3256
20
 
21
  ## Model description
22
 
 
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
+ - num_epochs: 20
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
+ | Training Loss | Epoch | Step | Validation Loss |
53
+ |:-------------:|:-------:|:----:|:---------------:|
54
+ | 4.595 | 0.9231 | 3 | 3.9759 |
55
+ | 4.0506 | 1.8462 | 6 | 3.4371 |
56
+ | 3.4359 | 2.7692 | 9 | 2.9322 |
57
+ | 2.1831 | 4.0 | 13 | 2.4351 |
58
+ | 2.4838 | 4.9231 | 16 | 2.1189 |
59
+ | 2.0545 | 5.8462 | 19 | 1.8192 |
60
+ | 1.7214 | 6.7692 | 22 | 1.6274 |
61
+ | 1.1606 | 8.0 | 26 | 1.4483 |
62
+ | 1.4113 | 8.9231 | 29 | 1.4009 |
63
+ | 1.3405 | 9.8462 | 32 | 1.3765 |
64
+ | 1.3347 | 10.7692 | 35 | 1.3599 |
65
+ | 0.9488 | 12.0 | 39 | 1.3411 |
66
+ | 1.2643 | 12.9231 | 42 | 1.3325 |
67
+ | 1.2198 | 13.8462 | 45 | 1.3287 |
68
+ | 1.2212 | 14.7692 | 48 | 1.3284 |
69
+ | 0.9143 | 16.0 | 52 | 1.3271 |
70
+ | 1.1902 | 16.9231 | 55 | 1.3265 |
71
+ | 1.173 | 17.8462 | 58 | 1.3258 |
72
+ | 0.8301 | 18.4615 | 60 | 1.3256 |
73
 
74
 
75
  ### Framework versions
adapter_config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
- "base_model_name_or_path": "TheBloke/Mistral-7B-Instruct-v0.2-GPTQ",
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
 
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
+ "base_model_name_or_path": null,
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c2ebfb58e971d3e9b92969fe11f4e01dda677c9762b47667080828bd60da0e63
3
- size 8397056
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63e9358af8c8cd87e6cb97512f2fbb8730317650474fd7e547deaeb5c84f4ac3
3
+ size 8400320
runs/Oct01_07-07-59_800808cfb2f8/events.out.tfevents.1727766479.800808cfb2f8.4374.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63105cb514a773bdd279d2f9ef7e9b826287572b0e883cc8a62847cfdcd73f6f
3
+ size 13013
runs/Oct01_07-20-08_800808cfb2f8/events.out.tfevents.1727767209.800808cfb2f8.4374.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b5ff3d25f23cefe139fc81b0e0d121738dbcb9d77d8b8e28794c80a37a6f337
3
+ size 9229
runs/Oct01_07-25-12_800808cfb2f8/events.out.tfevents.1727767513.800808cfb2f8.4374.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6ac7cbe1c87338ecfe4362b3c84503e6a91ecc306e231ee82809c8a457216fb
3
+ size 9229
runs/Oct01_07-29-40_800808cfb2f8/events.out.tfevents.1727767780.800808cfb2f8.4374.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1c8ca492aa0b74a976231abf0139f65384fb96ab4c0d2f38f2731d01c7efea8
3
+ size 8756
runs/Oct01_07-33-04_800808cfb2f8/events.out.tfevents.1727767984.800808cfb2f8.4374.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76fba110ffaeb3b62e3a817070db9ec4955920aadfcd09a953325261e9ad4a57
3
+ size 8756
runs/Oct01_07-36-27_800808cfb2f8/events.out.tfevents.1727768188.800808cfb2f8.4374.6 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4505f4ea0480a8192df037999ce36451ff97e91e52be369e74c4f5285df5ea52
3
+ size 9229
runs/Oct01_07-59-17_800808cfb2f8/events.out.tfevents.1727769557.800808cfb2f8.4374.7 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad4dd7cf5a98e73b76a66d3412964d1ccdd0fcbb15cec069f251ddd2cce7a33e
3
+ size 8756
runs/Oct01_08-03-03_800808cfb2f8/events.out.tfevents.1727769783.800808cfb2f8.4374.8 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:026ca15cc58a9349c953de65d7c0e157dd731ffb8ba70bea677f40a48a0f1383
3
+ size 9704
runs/Oct01_08-21-33_800808cfb2f8/events.out.tfevents.1727770894.800808cfb2f8.4374.9 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8023835d3b7ea8f654534df9de98399513668856a546e7c3f7b46964efdf6243
3
+ size 9702
runs/Oct01_08-28-24_800808cfb2f8/events.out.tfevents.1727771308.800808cfb2f8.4374.10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c30d855a1debc20f1e7db0b61585a1fd16bf49dc474cbbda42de41a6872b5301
3
+ size 5569
runs/Oct01_08-28-24_800808cfb2f8/events.out.tfevents.1727771323.800808cfb2f8.4374.11 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f99acc75a336b47726bfe7a159bffcf254c633e9d78109b205202614cdd05bc4
3
+ size 10647
runs/Oct01_08-36-25_800808cfb2f8/events.out.tfevents.1727771788.800808cfb2f8.4374.12 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98fa4b63d7817ffe62a669f3ada57cb104f630edabb48e317806318f1249c21f
3
+ size 10647
runs/Oct01_08-42-19_800808cfb2f8/events.out.tfevents.1727772139.800808cfb2f8.4374.13 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fca1782fa1cc8a85532830dde19dab65c1d8944928f06ef80f8f425d6937475
3
+ size 10648
runs/Oct01_08-48-12_800808cfb2f8/events.out.tfevents.1727772492.800808cfb2f8.4374.14 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:210e74c1105767f3411340cb3f2d6a4ed0161b305d32678b90204df71c53aaea
3
+ size 14904
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:77b0f20dc1fbb9c49cf28daab593bafe17e3e4070673710cb316beac6dcfffb1
3
  size 5176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7284c41d28a022d34e121cc49b896d919a36417b0f97627a8015debe2f329e5
3
  size 5176