lbourdois commited on
Commit
783b57e
·
verified ·
1 Parent(s): e9277f5

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +164 -150
README.md CHANGED
@@ -1,151 +1,165 @@
1
- ---
2
- library_name: peft
3
- license: apache-2.0
4
- base_model: Qwen/Qwen2.5-1.5B
5
- tags:
6
- - axolotl
7
- - generated_from_trainer
8
- model-index:
9
- - name: c103eab8-e475-40c2-9ec4-0fb40ae0f60e
10
- results: []
11
- ---
12
-
13
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
- should probably proofread and complete it, then remove this comment. -->
15
-
16
- [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
17
- <details><summary>See axolotl config</summary>
18
-
19
- axolotl version: `0.4.1`
20
- ```yaml
21
- adapter: lora
22
- base_model: Qwen/Qwen2.5-1.5B
23
- bf16: auto
24
- chat_template: llama3
25
- dataloader_num_workers: 12
26
- dataset_prepared_path: null
27
- datasets:
28
- - data_files:
29
- - 5459a8788029b49c_train_data.json
30
- ds_type: json
31
- format: custom
32
- path: /workspace/input_data/5459a8788029b49c_train_data.json
33
- type:
34
- field_input: sent_1
35
- field_instruction: original_l1
36
- field_output: sent_2
37
- format: '{instruction} {input}'
38
- no_input_format: '{instruction}'
39
- system_format: '{system}'
40
- system_prompt: ''
41
- debug: null
42
- deepspeed: null
43
- early_stopping_patience: null
44
- eval_max_new_tokens: 128
45
- eval_steps: null
46
- eval_table_size: null
47
- evals_per_epoch: null
48
- flash_attention: true
49
- fp16: null
50
- fsdp: null
51
- fsdp_config: null
52
- gradient_accumulation_steps: 8
53
- gradient_checkpointing: false
54
- group_by_length: false
55
- hub_model_id: prxy5604/c103eab8-e475-40c2-9ec4-0fb40ae0f60e
56
- hub_repo: null
57
- hub_strategy: checkpoint
58
- hub_token: null
59
- learning_rate: 0.0001
60
- load_in_4bit: false
61
- load_in_8bit: false
62
- local_rank: null
63
- logging_steps: null
64
- lora_alpha: 32
65
- lora_dropout: 0.15
66
- lora_fan_in_fan_out: null
67
- lora_model_dir: null
68
- lora_r: 32
69
- lora_target_linear: true
70
- loraplus_lr_embedding: 1.0e-06
71
- loraplus_lr_ratio: 16
72
- lr_scheduler: cosine
73
- max_grad_norm: 1
74
- max_steps: 3000
75
- micro_batch_size: 8
76
- mlflow_experiment_name: /tmp/5459a8788029b49c_train_data.json
77
- model_type: AutoModelForCausalLM
78
- num_epochs: 10
79
- optimizer: adamw_torch_fused
80
- output_dir: miner_id_24
81
- pad_to_sequence_len: true
82
- resume_from_checkpoint: null
83
- s2_attention: null
84
- sample_packing: false
85
- save_steps: null
86
- save_total_limit: 10
87
- saves_per_epoch: 0
88
- sequence_len: 1024
89
- strict: false
90
- tf32: true
91
- tokenizer_type: AutoTokenizer
92
- train_on_inputs: false
93
- trust_remote_code: true
94
- val_set_size: 0.0
95
- wandb_entity: null
96
- wandb_mode: online
97
- wandb_name: 4ff3bc45-7250-410b-9a91-04399ff26318
98
- wandb_project: Gradients-On-Demand
99
- wandb_run: your_name
100
- wandb_runid: 4ff3bc45-7250-410b-9a91-04399ff26318
101
- warmup_steps: 100
102
- weight_decay: 0
103
- xformers_attention: null
104
-
105
- ```
106
-
107
- </details><br>
108
-
109
- # c103eab8-e475-40c2-9ec4-0fb40ae0f60e
110
-
111
- This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on the None dataset.
112
-
113
- ## Model description
114
-
115
- More information needed
116
-
117
- ## Intended uses & limitations
118
-
119
- More information needed
120
-
121
- ## Training and evaluation data
122
-
123
- More information needed
124
-
125
- ## Training procedure
126
-
127
- ### Training hyperparameters
128
-
129
- The following hyperparameters were used during training:
130
- - learning_rate: 0.0001
131
- - train_batch_size: 8
132
- - eval_batch_size: 8
133
- - seed: 42
134
- - gradient_accumulation_steps: 8
135
- - total_train_batch_size: 64
136
- - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
137
- - lr_scheduler_type: cosine
138
- - lr_scheduler_warmup_steps: 100
139
- - training_steps: 3000
140
-
141
- ### Training results
142
-
143
-
144
-
145
- ### Framework versions
146
-
147
- - PEFT 0.13.2
148
- - Transformers 4.46.0
149
- - Pytorch 2.5.0+cu124
150
- - Datasets 3.0.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
151
  - Tokenizers 0.20.1
 
1
+ ---
2
+ library_name: peft
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen2.5-1.5B
5
+ tags:
6
+ - axolotl
7
+ - generated_from_trainer
8
+ language:
9
+ - zho
10
+ - eng
11
+ - fra
12
+ - spa
13
+ - por
14
+ - deu
15
+ - ita
16
+ - rus
17
+ - jpn
18
+ - kor
19
+ - vie
20
+ - tha
21
+ - ara
22
+ model-index:
23
+ - name: c103eab8-e475-40c2-9ec4-0fb40ae0f60e
24
+ results: []
25
+ ---
26
+
27
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
28
+ should probably proofread and complete it, then remove this comment. -->
29
+
30
+ [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
31
+ <details><summary>See axolotl config</summary>
32
+
33
+ axolotl version: `0.4.1`
34
+ ```yaml
35
+ adapter: lora
36
+ base_model: Qwen/Qwen2.5-1.5B
37
+ bf16: auto
38
+ chat_template: llama3
39
+ dataloader_num_workers: 12
40
+ dataset_prepared_path: null
41
+ datasets:
42
+ - data_files:
43
+ - 5459a8788029b49c_train_data.json
44
+ ds_type: json
45
+ format: custom
46
+ path: /workspace/input_data/5459a8788029b49c_train_data.json
47
+ type:
48
+ field_input: sent_1
49
+ field_instruction: original_l1
50
+ field_output: sent_2
51
+ format: '{instruction} {input}'
52
+ no_input_format: '{instruction}'
53
+ system_format: '{system}'
54
+ system_prompt: ''
55
+ debug: null
56
+ deepspeed: null
57
+ early_stopping_patience: null
58
+ eval_max_new_tokens: 128
59
+ eval_steps: null
60
+ eval_table_size: null
61
+ evals_per_epoch: null
62
+ flash_attention: true
63
+ fp16: null
64
+ fsdp: null
65
+ fsdp_config: null
66
+ gradient_accumulation_steps: 8
67
+ gradient_checkpointing: false
68
+ group_by_length: false
69
+ hub_model_id: prxy5604/c103eab8-e475-40c2-9ec4-0fb40ae0f60e
70
+ hub_repo: null
71
+ hub_strategy: checkpoint
72
+ hub_token: null
73
+ learning_rate: 0.0001
74
+ load_in_4bit: false
75
+ load_in_8bit: false
76
+ local_rank: null
77
+ logging_steps: null
78
+ lora_alpha: 32
79
+ lora_dropout: 0.15
80
+ lora_fan_in_fan_out: null
81
+ lora_model_dir: null
82
+ lora_r: 32
83
+ lora_target_linear: true
84
+ loraplus_lr_embedding: 1.0e-06
85
+ loraplus_lr_ratio: 16
86
+ lr_scheduler: cosine
87
+ max_grad_norm: 1
88
+ max_steps: 3000
89
+ micro_batch_size: 8
90
+ mlflow_experiment_name: /tmp/5459a8788029b49c_train_data.json
91
+ model_type: AutoModelForCausalLM
92
+ num_epochs: 10
93
+ optimizer: adamw_torch_fused
94
+ output_dir: miner_id_24
95
+ pad_to_sequence_len: true
96
+ resume_from_checkpoint: null
97
+ s2_attention: null
98
+ sample_packing: false
99
+ save_steps: null
100
+ save_total_limit: 10
101
+ saves_per_epoch: 0
102
+ sequence_len: 1024
103
+ strict: false
104
+ tf32: true
105
+ tokenizer_type: AutoTokenizer
106
+ train_on_inputs: false
107
+ trust_remote_code: true
108
+ val_set_size: 0.0
109
+ wandb_entity: null
110
+ wandb_mode: online
111
+ wandb_name: 4ff3bc45-7250-410b-9a91-04399ff26318
112
+ wandb_project: Gradients-On-Demand
113
+ wandb_run: your_name
114
+ wandb_runid: 4ff3bc45-7250-410b-9a91-04399ff26318
115
+ warmup_steps: 100
116
+ weight_decay: 0
117
+ xformers_attention: null
118
+
119
+ ```
120
+
121
+ </details><br>
122
+
123
+ # c103eab8-e475-40c2-9ec4-0fb40ae0f60e
124
+
125
+ This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on the None dataset.
126
+
127
+ ## Model description
128
+
129
+ More information needed
130
+
131
+ ## Intended uses & limitations
132
+
133
+ More information needed
134
+
135
+ ## Training and evaluation data
136
+
137
+ More information needed
138
+
139
+ ## Training procedure
140
+
141
+ ### Training hyperparameters
142
+
143
+ The following hyperparameters were used during training:
144
+ - learning_rate: 0.0001
145
+ - train_batch_size: 8
146
+ - eval_batch_size: 8
147
+ - seed: 42
148
+ - gradient_accumulation_steps: 8
149
+ - total_train_batch_size: 64
150
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
151
+ - lr_scheduler_type: cosine
152
+ - lr_scheduler_warmup_steps: 100
153
+ - training_steps: 3000
154
+
155
+ ### Training results
156
+
157
+
158
+
159
+ ### Framework versions
160
+
161
+ - PEFT 0.13.2
162
+ - Transformers 4.46.0
163
+ - Pytorch 2.5.0+cu124
164
+ - Datasets 3.0.1
165
  - Tokenizers 0.20.1