Aivesa commited on
Commit
80bfabf
·
verified ·
1 Parent(s): 69a853d

End of training

Browse files
Files changed (1) hide show
  1. README.md +16 -19
README.md CHANGED
@@ -1,14 +1,14 @@
1
  ---
2
  library_name: peft
3
  license: apache-2.0
4
- base_model: openlm-research/open_llama_3b
5
  tags:
6
  - axolotl
7
  - generated_from_trainer
8
  datasets:
9
- - Aivesa/dataset_b98c9df0-5280-40e9-9125-37aafc6279e9
10
  model-index:
11
- - name: cd10faed-0ece-4718-b1de-7fa9782b2746
12
  results: []
13
  ---
14
 
@@ -21,18 +21,17 @@ should probably proofread and complete it, then remove this comment. -->
21
  axolotl version: `0.6.0`
22
  ```yaml
23
  adapter: lora
24
- base_model: openlm-research/open_llama_3b
25
  bf16: auto
26
  chat_template: llama3
27
  dataset_prepared_path: /workspace/axolotl/data/prepared
28
  datasets:
29
  - ds_type: json
30
  format: custom
31
- path: Aivesa/dataset_b98c9df0-5280-40e9-9125-37aafc6279e9
32
  type:
33
- field_input: sentence1
34
- field_instruction: genre
35
- field_output: sentence2
36
  system_format: '{system}'
37
  system_prompt: ''
38
  debug: null
@@ -48,7 +47,7 @@ fsdp_config: null
48
  gradient_accumulation_steps: 4
49
  gradient_checkpointing: false
50
  group_by_length: false
51
- hub_model_id: Aivesa/cd10faed-0ece-4718-b1de-7fa9782b2746
52
  hub_private_repo: true
53
  hub_repo: null
54
  hub_strategy: checkpoint
@@ -79,8 +78,6 @@ sample_packing: false
79
  save_safetensors: true
80
  saves_per_epoch: 4
81
  sequence_len: 512
82
- special_tokens:
83
- pad_token: </s>
84
  strict: false
85
  tf32: false
86
  tokenizer_type: AutoTokenizer
@@ -90,10 +87,10 @@ use_accelerate: true
90
  val_set_size: 0.05
91
  wandb_entity: null
92
  wandb_mode: online
93
- wandb_name: b98c9df0-5280-40e9-9125-37aafc6279e9
94
  wandb_project: Gradients-On-Demand
95
  wandb_run: your_name
96
- wandb_runid: b98c9df0-5280-40e9-9125-37aafc6279e9
97
  warmup_steps: 10
98
  weight_decay: 0.0
99
  xformers_attention: null
@@ -102,11 +99,11 @@ xformers_attention: null
102
 
103
  </details><br>
104
 
105
- # cd10faed-0ece-4718-b1de-7fa9782b2746
106
 
107
- This model is a fine-tuned version of [openlm-research/open_llama_3b](https://huggingface.co/openlm-research/open_llama_3b) on the Aivesa/dataset_b98c9df0-5280-40e9-9125-37aafc6279e9 dataset.
108
  It achieves the following results on the evaluation set:
109
- - Loss: 0.9438
110
 
111
  ## Model description
112
 
@@ -140,9 +137,9 @@ The following hyperparameters were used during training:
140
 
141
  | Training Loss | Epoch | Step | Validation Loss |
142
  |:-------------:|:------:|:----:|:---------------:|
143
- | 0.9494 | 0.0030 | 3 | 1.0563 |
144
- | 1.1032 | 0.0060 | 6 | 1.0397 |
145
- | 0.9986 | 0.0090 | 9 | 0.9438 |
146
 
147
 
148
  ### Framework versions
 
1
  ---
2
  library_name: peft
3
  license: apache-2.0
4
+ base_model: unsloth/Qwen2.5-1.5B
5
  tags:
6
  - axolotl
7
  - generated_from_trainer
8
  datasets:
9
+ - Aivesa/dataset_b958f22f-e98e-4433-9870-28fc65ec5af5
10
  model-index:
11
+ - name: 553c1bc9-598d-4fbe-9731-9a88e1324be5
12
  results: []
13
  ---
14
 
 
21
  axolotl version: `0.6.0`
22
  ```yaml
23
  adapter: lora
24
+ base_model: unsloth/Qwen2.5-1.5B
25
  bf16: auto
26
  chat_template: llama3
27
  dataset_prepared_path: /workspace/axolotl/data/prepared
28
  datasets:
29
  - ds_type: json
30
  format: custom
31
+ path: Aivesa/dataset_b958f22f-e98e-4433-9870-28fc65ec5af5
32
  type:
33
+ field_instruction: user
34
+ field_output: assistant
 
35
  system_format: '{system}'
36
  system_prompt: ''
37
  debug: null
 
47
  gradient_accumulation_steps: 4
48
  gradient_checkpointing: false
49
  group_by_length: false
50
+ hub_model_id: Aivesa/553c1bc9-598d-4fbe-9731-9a88e1324be5
51
  hub_private_repo: true
52
  hub_repo: null
53
  hub_strategy: checkpoint
 
78
  save_safetensors: true
79
  saves_per_epoch: 4
80
  sequence_len: 512
 
 
81
  strict: false
82
  tf32: false
83
  tokenizer_type: AutoTokenizer
 
87
  val_set_size: 0.05
88
  wandb_entity: null
89
  wandb_mode: online
90
+ wandb_name: b958f22f-e98e-4433-9870-28fc65ec5af5
91
  wandb_project: Gradients-On-Demand
92
  wandb_run: your_name
93
+ wandb_runid: b958f22f-e98e-4433-9870-28fc65ec5af5
94
  warmup_steps: 10
95
  weight_decay: 0.0
96
  xformers_attention: null
 
99
 
100
  </details><br>
101
 
102
+ # 553c1bc9-598d-4fbe-9731-9a88e1324be5
103
 
104
+ This model is a fine-tuned version of [unsloth/Qwen2.5-1.5B](https://huggingface.co/unsloth/Qwen2.5-1.5B) on the Aivesa/dataset_b958f22f-e98e-4433-9870-28fc65ec5af5 dataset.
105
  It achieves the following results on the evaluation set:
106
+ - Loss: 1.0034
107
 
108
  ## Model description
109
 
 
137
 
138
  | Training Loss | Epoch | Step | Validation Loss |
139
  |:-------------:|:------:|:----:|:---------------:|
140
+ | 1.3033 | 0.0006 | 3 | 1.2600 |
141
+ | 1.2558 | 0.0013 | 6 | 1.1808 |
142
+ | 1.0792 | 0.0019 | 9 | 1.0034 |
143
 
144
 
145
  ### Framework versions