End of training
Browse files- README.md +11 -4
- adapter_model.bin +1 -1
README.md
CHANGED
@@ -65,7 +65,7 @@ lora_model_dir: null
|
|
65 |
lora_r: 8
|
66 |
lora_target_linear: true
|
67 |
lr_scheduler: cosine
|
68 |
-
max_steps:
|
69 |
micro_batch_size: 2
|
70 |
mlflow_experiment_name: /tmp/43b97beb6cb95c3f_train_data.json
|
71 |
model_type: AutoModelForCausalLM
|
@@ -92,7 +92,7 @@ wandb_name: cc30fccd-d4cd-4973-81b5-20778b16cec3
|
|
92 |
wandb_project: Gradients-On-Demand
|
93 |
wandb_run: your_name
|
94 |
wandb_runid: cc30fccd-d4cd-4973-81b5-20778b16cec3
|
95 |
-
warmup_steps:
|
96 |
weight_decay: 0.0
|
97 |
xformers_attention: null
|
98 |
|
@@ -103,6 +103,8 @@ xformers_attention: null
|
|
103 |
# 86758f57-249e-432a-b11e-8e003113b4dc
|
104 |
|
105 |
This model is a fine-tuned version of [OpenBuddy/openbuddy-llama2-13b-v8.1-fp16](https://huggingface.co/OpenBuddy/openbuddy-llama2-13b-v8.1-fp16) on the None dataset.
|
|
|
|
|
106 |
|
107 |
## Model description
|
108 |
|
@@ -129,14 +131,19 @@ The following hyperparameters were used during training:
|
|
129 |
- total_train_batch_size: 8
|
130 |
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
131 |
- lr_scheduler_type: cosine
|
132 |
-
- lr_scheduler_warmup_steps:
|
133 |
-
- training_steps:
|
134 |
|
135 |
### Training results
|
136 |
|
137 |
| Training Loss | Epoch | Step | Validation Loss |
|
138 |
|:-------------:|:------:|:----:|:---------------:|
|
139 |
| No log | 0.0005 | 1 | 1.0615 |
|
|
|
|
|
|
|
|
|
|
|
140 |
|
141 |
|
142 |
### Framework versions
|
|
|
65 |
lora_r: 8
|
66 |
lora_target_linear: true
|
67 |
lr_scheduler: cosine
|
68 |
+
max_steps: 50
|
69 |
micro_batch_size: 2
|
70 |
mlflow_experiment_name: /tmp/43b97beb6cb95c3f_train_data.json
|
71 |
model_type: AutoModelForCausalLM
|
|
|
92 |
wandb_project: Gradients-On-Demand
|
93 |
wandb_run: your_name
|
94 |
wandb_runid: cc30fccd-d4cd-4973-81b5-20778b16cec3
|
95 |
+
warmup_steps: 10
|
96 |
weight_decay: 0.0
|
97 |
xformers_attention: null
|
98 |
|
|
|
103 |
# 86758f57-249e-432a-b11e-8e003113b4dc
|
104 |
|
105 |
This model is a fine-tuned version of [OpenBuddy/openbuddy-llama2-13b-v8.1-fp16](https://huggingface.co/OpenBuddy/openbuddy-llama2-13b-v8.1-fp16) on the None dataset.
|
106 |
+
It achieves the following results on the evaluation set:
|
107 |
+
- Loss: 0.0107
|
108 |
|
109 |
## Model description
|
110 |
|
|
|
131 |
- total_train_batch_size: 8
|
132 |
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
133 |
- lr_scheduler_type: cosine
|
134 |
+
- lr_scheduler_warmup_steps: 10
|
135 |
+
- training_steps: 50
|
136 |
|
137 |
### Training results
|
138 |
|
139 |
| Training Loss | Epoch | Step | Validation Loss |
|
140 |
|:-------------:|:------:|:----:|:---------------:|
|
141 |
| No log | 0.0005 | 1 | 1.0615 |
|
142 |
+
| 0.727 | 0.0053 | 10 | 0.3960 |
|
143 |
+
| 0.093 | 0.0107 | 20 | 0.0437 |
|
144 |
+
| 0.0159 | 0.0160 | 30 | 0.0113 |
|
145 |
+
| 0.0064 | 0.0213 | 40 | 0.0113 |
|
146 |
+
| 0.0082 | 0.0267 | 50 | 0.0107 |
|
147 |
|
148 |
|
149 |
### Framework versions
|
adapter_model.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 125375434
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:a9c9a9966b8ad17948c25d896039ad79c567fab4d8ff4252b4649805490022b1
|
3 |
size 125375434
|