cpheemagazine commited on
Commit
f9a5b87
·
verified ·
1 Parent(s): 3ec34cf

End of training

Browse files
Files changed (1) hide show
  1. README.md +121 -0
README.md ADDED
@@ -0,0 +1,121 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen2.5-1.5B
5
+ tags:
6
+ - axolotl
7
+ - generated_from_trainer
8
+ model-index:
9
+ - name: 3944d5cd-8236-424a-b93e-9d49250e4314
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
17
+ <details><summary>See axolotl config</summary>
18
+
19
+ axolotl version: `0.10.0.dev0`
20
+ ```yaml
21
+ adapter: lora
22
+ base_model: Qwen/Qwen2.5-1.5B
23
+ bf16: true
24
+ datasets:
25
+ - data_files:
26
+ - c8f334387c1e0ed5_train_data.json
27
+ ds_type: json
28
+ format: custom
29
+ path: /workspace/input_data/
30
+ type:
31
+ field_instruction: instruct
32
+ field_output: output
33
+ format: '{instruction}'
34
+ no_input_format: '{instruction}'
35
+ system_format: '{system}'
36
+ system_prompt: ''
37
+ eval_max_new_tokens: 128
38
+ evals_per_epoch: 4
39
+ flash_attention: false
40
+ fp16: false
41
+ gradient_accumulation_steps: 1
42
+ gradient_checkpointing: true
43
+ group_by_length: true
44
+ hf_upload_public: true
45
+ hf_upload_repo_type: model
46
+ hub_model_id: cpheemagazine/3944d5cd-8236-424a-b93e-9d49250e4314
47
+ learning_rate: 0.0002
48
+ load_in_4bit: false
49
+ logging_steps: 10
50
+ lora_alpha: 16
51
+ lora_dropout: 0.05
52
+ lora_fan_in_fan_out: false
53
+ lora_r: 8
54
+ lora_target_linear: true
55
+ lr_scheduler: cosine
56
+ max_steps: 874
57
+ micro_batch_size: 36
58
+ mlflow_experiment_name: /tmp/c8f334387c1e0ed5_train_data.json
59
+ optimizer: adamw_torch_fused
60
+ output_dir: miner_id_24
61
+ rl: null
62
+ sample_packing: true
63
+ save_steps: 131
64
+ sequence_len: 2048
65
+ tf32: true
66
+ tokenizer_type: AutoTokenizer
67
+ train_on_inputs: true
68
+ trl: null
69
+ trust_remote_code: true
70
+ wandb_name: b91c504d-65c2-43bc-8b29-5e9c0920f259
71
+ wandb_project: Gradients-On-Demand
72
+ wandb_run: apriasmoro
73
+ wandb_runid: b91c504d-65c2-43bc-8b29-5e9c0920f259
74
+ warmup_steps: 87
75
+ weight_decay: 0.02
76
+
77
+ ```
78
+
79
+ </details><br>
80
+
81
+ # 3944d5cd-8236-424a-b93e-9d49250e4314
82
+
83
+ This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on an unknown dataset.
84
+
85
+ ## Model description
86
+
87
+ More information needed
88
+
89
+ ## Intended uses & limitations
90
+
91
+ More information needed
92
+
93
+ ## Training and evaluation data
94
+
95
+ More information needed
96
+
97
+ ## Training procedure
98
+
99
+ ### Training hyperparameters
100
+
101
+ The following hyperparameters were used during training:
102
+ - learning_rate: 0.0002
103
+ - train_batch_size: 36
104
+ - eval_batch_size: 36
105
+ - seed: 42
106
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
107
+ - lr_scheduler_type: cosine
108
+ - lr_scheduler_warmup_steps: 87
109
+ - training_steps: 874
110
+
111
+ ### Training results
112
+
113
+
114
+
115
+ ### Framework versions
116
+
117
+ - PEFT 0.15.2
118
+ - Transformers 4.51.3
119
+ - Pytorch 2.5.1+cu124
120
+ - Datasets 3.5.1
121
+ - Tokenizers 0.21.1