lbourdois commited on
Commit
5d18c1b
·
verified ·
1 Parent(s): a2532ba

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +204 -194
README.md CHANGED
@@ -1,194 +1,204 @@
1
- ---
2
- language:
3
- - fr
4
- - en
5
- license: other
6
- library_name: transformers
7
- tags:
8
- - chat
9
- - qwen
10
- - qwen2.5
11
- - finetune
12
- - french
13
- - english
14
- base_model: Qwen/Qwen2.5-3B
15
- datasets:
16
- - MaziyarPanahi/french_instruct_sharegpt
17
- - arcee-ai/EvolKit-20k
18
- model_name: calme-3.2-instruct-3b
19
- license_name: qwen-research
20
- license_link: https://huggingface.co/Qwen/Qwen2.5-3B/blob/main/LICENSE
21
- pipeline_tag: text-generation
22
- inference: false
23
- model_creator: MaziyarPanahi
24
- quantized_by: MaziyarPanahi
25
- model-index:
26
- - name: calme-3.2-instruct-3b
27
- results:
28
- - task:
29
- type: text-generation
30
- name: Text Generation
31
- dataset:
32
- name: IFEval (0-Shot)
33
- type: HuggingFaceH4/ifeval
34
- args:
35
- num_few_shot: 0
36
- metrics:
37
- - type: inst_level_strict_acc and prompt_level_strict_acc
38
- value: 55.33
39
- name: strict accuracy
40
- source:
41
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
42
- name: Open LLM Leaderboard
43
- - task:
44
- type: text-generation
45
- name: Text Generation
46
- dataset:
47
- name: BBH (3-Shot)
48
- type: BBH
49
- args:
50
- num_few_shot: 3
51
- metrics:
52
- - type: acc_norm
53
- value: 27.98
54
- name: normalized accuracy
55
- source:
56
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
57
- name: Open LLM Leaderboard
58
- - task:
59
- type: text-generation
60
- name: Text Generation
61
- dataset:
62
- name: MATH Lvl 5 (4-Shot)
63
- type: hendrycks/competition_math
64
- args:
65
- num_few_shot: 4
66
- metrics:
67
- - type: exact_match
68
- value: 9.89
69
- name: exact match
70
- source:
71
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
72
- name: Open LLM Leaderboard
73
- - task:
74
- type: text-generation
75
- name: Text Generation
76
- dataset:
77
- name: GPQA (0-shot)
78
- type: Idavidrein/gpqa
79
- args:
80
- num_few_shot: 0
81
- metrics:
82
- - type: acc_norm
83
- value: 4.47
84
- name: acc_norm
85
- source:
86
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
87
- name: Open LLM Leaderboard
88
- - task:
89
- type: text-generation
90
- name: Text Generation
91
- dataset:
92
- name: MuSR (0-shot)
93
- type: TAUR-Lab/MuSR
94
- args:
95
- num_few_shot: 0
96
- metrics:
97
- - type: acc_norm
98
- value: 8.79
99
- name: acc_norm
100
- source:
101
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
102
- name: Open LLM Leaderboard
103
- - task:
104
- type: text-generation
105
- name: Text Generation
106
- dataset:
107
- name: MMLU-PRO (5-shot)
108
- type: TIGER-Lab/MMLU-Pro
109
- config: main
110
- split: test
111
- args:
112
- num_few_shot: 5
113
- metrics:
114
- - type: acc
115
- value: 29.48
116
- name: accuracy
117
- source:
118
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
119
- name: Open LLM Leaderboard
120
- ---
121
-
122
- <img src="./calme_3.png" alt="Calme-3 Models" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
123
-
124
- > [!TIP]
125
- > This is avery small model, so it might not perform well for some prompts and may be sensitive to hyper parameters. I would appreciate any feedback to see if I can fix any issues in the next iteration. ❤️
126
- >
127
- # MaziyarPanahi/calme-3.2-instruct-3b
128
-
129
- This model is an advanced iteration of the powerful `Qwen/Qwen2.5-3B`, specifically fine-tuned to enhance its capabilities in generic domains.
130
-
131
-
132
- # Quantized GGUF
133
-
134
- All GGUF models are available here: [MaziyarPanahi/calme-3.2-instruct-3b-GGUF](https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-3b-GGUF)
135
-
136
-
137
- # 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
138
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__calme-3.2-instruct-3b)
139
-
140
- | Metric |Value|
141
- |-------------------|----:|
142
- |Avg. |22.66|
143
- |IFEval (0-Shot) |55.33|
144
- |BBH (3-Shot) |27.98|
145
- |MATH Lvl 5 (4-Shot)| 9.89|
146
- |GPQA (0-shot) | 4.47|
147
- |MuSR (0-shot) | 8.79|
148
- |MMLU-PRO (5-shot) |29.48|
149
-
150
-
151
- # Prompt Template
152
-
153
- This model uses `ChatML` prompt template:
154
-
155
- ```
156
- <|im_start|>system
157
- {System}
158
- <|im_end|>
159
- <|im_start|>user
160
- {User}
161
- <|im_end|>
162
- <|im_start|>assistant
163
- {Assistant}
164
- ````
165
-
166
- # How to use
167
-
168
-
169
- ```python
170
-
171
- # Use a pipeline as a high-level helper
172
-
173
- from transformers import pipeline
174
-
175
- messages = [
176
- {"role": "user", "content": "Who are you?"},
177
- ]
178
- pipe = pipeline("text-generation", model="MaziyarPanahi/calme-3.2-instruct-3b")
179
- pipe(messages)
180
-
181
-
182
- # Load model directly
183
-
184
- from transformers import AutoTokenizer, AutoModelForCausalLM
185
-
186
- tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/calme-3.2-instruct-3b")
187
- model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/calme-3.2-instruct-3b")
188
- ```
189
-
190
-
191
-
192
- # Ethical Considerations
193
-
194
- As with any large language model, users should be aware of potential biases and limitations. We recommend implementing appropriate safeguards and human oversight when deploying this model in production environments.
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zho
4
+ - eng
5
+ - fra
6
+ - spa
7
+ - por
8
+ - deu
9
+ - ita
10
+ - rus
11
+ - jpn
12
+ - kor
13
+ - vie
14
+ - tha
15
+ - ara
16
+ license: other
17
+ library_name: transformers
18
+ tags:
19
+ - chat
20
+ - qwen
21
+ - qwen2.5
22
+ - finetune
23
+ - french
24
+ - english
25
+ base_model: Qwen/Qwen2.5-3B
26
+ datasets:
27
+ - MaziyarPanahi/french_instruct_sharegpt
28
+ - arcee-ai/EvolKit-20k
29
+ license_name: qwen-research
30
+ license_link: https://huggingface.co/Qwen/Qwen2.5-3B/blob/main/LICENSE
31
+ pipeline_tag: text-generation
32
+ inference: false
33
+ model_creator: MaziyarPanahi
34
+ quantized_by: MaziyarPanahi
35
+ model-index:
36
+ - name: calme-3.2-instruct-3b
37
+ results:
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: IFEval (0-Shot)
43
+ type: HuggingFaceH4/ifeval
44
+ args:
45
+ num_few_shot: 0
46
+ metrics:
47
+ - type: inst_level_strict_acc and prompt_level_strict_acc
48
+ value: 55.33
49
+ name: strict accuracy
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: BBH (3-Shot)
58
+ type: BBH
59
+ args:
60
+ num_few_shot: 3
61
+ metrics:
62
+ - type: acc_norm
63
+ value: 27.98
64
+ name: normalized accuracy
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: MATH Lvl 5 (4-Shot)
73
+ type: hendrycks/competition_math
74
+ args:
75
+ num_few_shot: 4
76
+ metrics:
77
+ - type: exact_match
78
+ value: 9.89
79
+ name: exact match
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: GPQA (0-shot)
88
+ type: Idavidrein/gpqa
89
+ args:
90
+ num_few_shot: 0
91
+ metrics:
92
+ - type: acc_norm
93
+ value: 4.47
94
+ name: acc_norm
95
+ source:
96
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
97
+ name: Open LLM Leaderboard
98
+ - task:
99
+ type: text-generation
100
+ name: Text Generation
101
+ dataset:
102
+ name: MuSR (0-shot)
103
+ type: TAUR-Lab/MuSR
104
+ args:
105
+ num_few_shot: 0
106
+ metrics:
107
+ - type: acc_norm
108
+ value: 8.79
109
+ name: acc_norm
110
+ source:
111
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
112
+ name: Open LLM Leaderboard
113
+ - task:
114
+ type: text-generation
115
+ name: Text Generation
116
+ dataset:
117
+ name: MMLU-PRO (5-shot)
118
+ type: TIGER-Lab/MMLU-Pro
119
+ config: main
120
+ split: test
121
+ args:
122
+ num_few_shot: 5
123
+ metrics:
124
+ - type: acc
125
+ value: 29.48
126
+ name: accuracy
127
+ source:
128
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-3.2-instruct-3b
129
+ name: Open LLM Leaderboard
130
+ ---
131
+
132
+ <img src="./calme_3.png" alt="Calme-3 Models" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
133
+
134
+ > [!TIP]
135
+ > This is avery small model, so it might not perform well for some prompts and may be sensitive to hyper parameters. I would appreciate any feedback to see if I can fix any issues in the next iteration. ❤️
136
+ >
137
+ # MaziyarPanahi/calme-3.2-instruct-3b
138
+
139
+ This model is an advanced iteration of the powerful `Qwen/Qwen2.5-3B`, specifically fine-tuned to enhance its capabilities in generic domains.
140
+
141
+
142
+ # ⚡ Quantized GGUF
143
+
144
+ All GGUF models are available here: [MaziyarPanahi/calme-3.2-instruct-3b-GGUF](https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-3b-GGUF)
145
+
146
+
147
+ # 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
148
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__calme-3.2-instruct-3b)
149
+
150
+ | Metric |Value|
151
+ |-------------------|----:|
152
+ |Avg. |22.66|
153
+ |IFEval (0-Shot) |55.33|
154
+ |BBH (3-Shot) |27.98|
155
+ |MATH Lvl 5 (4-Shot)| 9.89|
156
+ |GPQA (0-shot) | 4.47|
157
+ |MuSR (0-shot) | 8.79|
158
+ |MMLU-PRO (5-shot) |29.48|
159
+
160
+
161
+ # Prompt Template
162
+
163
+ This model uses `ChatML` prompt template:
164
+
165
+ ```
166
+ <|im_start|>system
167
+ {System}
168
+ <|im_end|>
169
+ <|im_start|>user
170
+ {User}
171
+ <|im_end|>
172
+ <|im_start|>assistant
173
+ {Assistant}
174
+ ````
175
+
176
+ # How to use
177
+
178
+
179
+ ```python
180
+
181
+ # Use a pipeline as a high-level helper
182
+
183
+ from transformers import pipeline
184
+
185
+ messages = [
186
+ {"role": "user", "content": "Who are you?"},
187
+ ]
188
+ pipe = pipeline("text-generation", model="MaziyarPanahi/calme-3.2-instruct-3b")
189
+ pipe(messages)
190
+
191
+
192
+ # Load model directly
193
+
194
+ from transformers import AutoTokenizer, AutoModelForCausalLM
195
+
196
+ tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/calme-3.2-instruct-3b")
197
+ model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/calme-3.2-instruct-3b")
198
+ ```
199
+
200
+
201
+
202
+ # Ethical Considerations
203
+
204
+ As with any large language model, users should be aware of potential biases and limitations. We recommend implementing appropriate safeguards and human oversight when deploying this model in production environments.