lbourdois commited on
Commit
7e2f0d5
·
verified ·
1 Parent(s): 5518976

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +229 -218
README.md CHANGED
@@ -1,218 +1,229 @@
1
- ---
2
- license: apache-2.0
3
- language:
4
- - en
5
- - zh
6
- base_model:
7
- - Qwen/Qwen2.5-14B-Instruct
8
- pipeline_tag: text-generation
9
- library_name: transformers
10
- tags:
11
- - text-generation-inference
12
- - math
13
- - code
14
- - trl
15
- - sft
16
- model-index:
17
- - name: Epimetheus-14B-Axo
18
- results:
19
- - task:
20
- type: text-generation
21
- name: Text Generation
22
- dataset:
23
- name: IFEval (0-Shot)
24
- type: wis-k/instruction-following-eval
25
- split: train
26
- args:
27
- num_few_shot: 0
28
- metrics:
29
- - type: inst_level_strict_acc and prompt_level_strict_acc
30
- value: 55.46
31
- name: averaged accuracy
32
- source:
33
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
34
- name: Open LLM Leaderboard
35
- - task:
36
- type: text-generation
37
- name: Text Generation
38
- dataset:
39
- name: BBH (3-Shot)
40
- type: SaylorTwift/bbh
41
- split: test
42
- args:
43
- num_few_shot: 3
44
- metrics:
45
- - type: acc_norm
46
- value: 51.46
47
- name: normalized accuracy
48
- source:
49
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
50
- name: Open LLM Leaderboard
51
- - task:
52
- type: text-generation
53
- name: Text Generation
54
- dataset:
55
- name: MATH Lvl 5 (4-Shot)
56
- type: lighteval/MATH-Hard
57
- split: test
58
- args:
59
- num_few_shot: 4
60
- metrics:
61
- - type: exact_match
62
- value: 41.01
63
- name: exact match
64
- source:
65
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
66
- name: Open LLM Leaderboard
67
- - task:
68
- type: text-generation
69
- name: Text Generation
70
- dataset:
71
- name: GPQA (0-shot)
72
- type: Idavidrein/gpqa
73
- split: train
74
- args:
75
- num_few_shot: 0
76
- metrics:
77
- - type: acc_norm
78
- value: 19.02
79
- name: acc_norm
80
- source:
81
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
82
- name: Open LLM Leaderboard
83
- - task:
84
- type: text-generation
85
- name: Text Generation
86
- dataset:
87
- name: MuSR (0-shot)
88
- type: TAUR-Lab/MuSR
89
- args:
90
- num_few_shot: 0
91
- metrics:
92
- - type: acc_norm
93
- value: 19.71
94
- name: acc_norm
95
- source:
96
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
97
- name: Open LLM Leaderboard
98
- - task:
99
- type: text-generation
100
- name: Text Generation
101
- dataset:
102
- name: MMLU-PRO (5-shot)
103
- type: TIGER-Lab/MMLU-Pro
104
- config: main
105
- split: test
106
- args:
107
- num_few_shot: 5
108
- metrics:
109
- - type: acc
110
- value: 47.82
111
- name: accuracy
112
- source:
113
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
114
- name: Open LLM Leaderboard
115
- ---
116
- ![aszsdcazss.png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/jU62zQyKIuTfwXewv4T0Y.png)
117
-
118
- # **Epimetheus-14B-Axo**
119
-
120
- > Epimetheus-14B-Axo is based on the Qwen 2.5 14B modality architecture, designed to enhance the reasoning capabilities of 14B-parameter models. This model is optimized for general-purpose reasoning and answering, excelling in contextual understanding, logical deduction, and multi-step problem-solving. It has been fine-tuned using a long chain-of-thought reasoning model and specialized datasets to improve comprehension, structured responses, and conversational intelligence.
121
-
122
- ## **Key Improvements**
123
- 1. **Enhanced General Knowledge**: The model provides broad knowledge across various domains, improving capabilities in answering questions accurately and generating coherent responses.
124
- 2. **Improved Instruction Following**: Significant advancements in understanding and following complex instructions, generating structured responses, and maintaining coherence over extended interactions.
125
- 3. **Versatile Adaptability**: More resilient to diverse prompts, enhancing its ability to handle a wide range of topics and conversation styles, including open-ended and structured inquiries.
126
- 4. **Long-Context Support**: Supports up to 128K tokens for input context and can generate up to 8K tokens in a single output, making it ideal for detailed responses.
127
- 5. **Multilingual Proficiency**: Supports over 29 languages, including English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
128
-
129
- ## **Quickstart with transformers**
130
-
131
- Here is a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and generate content:
132
-
133
- ```python
134
- from transformers import AutoModelForCausalLM, AutoTokenizer
135
-
136
- model_name = "prithivMLmods/Epimetheus-14B-Axo"
137
-
138
- model = AutoModelForCausalLM.from_pretrained(
139
- model_name,
140
- torch_dtype="auto",
141
- device_map="auto"
142
- )
143
- tokenizer = AutoTokenizer.from_pretrained(model_name)
144
-
145
- prompt = "What are the key principles of general-purpose AI?"
146
- messages = [
147
- {"role": "system", "content": "You are a helpful assistant capable of answering a wide range of questions."},
148
- {"role": "user", "content": prompt}
149
- ]
150
- text = tokenizer.apply_chat_template(
151
- messages,
152
- tokenize=False,
153
- add_generation_prompt=True
154
- )
155
- model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
156
-
157
- generated_ids = model.generate(
158
- **model_inputs,
159
- max_new_tokens=512
160
- )
161
- generated_ids = [
162
- output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
163
- ]
164
-
165
- response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
166
- ```
167
-
168
- ## **Intended Use**
169
- 1. **General-Purpose Reasoning**:
170
- Designed for broad applicability, assisting with logical reasoning, answering diverse questions, and solving general knowledge problems.
171
-
172
- 2. **Educational and Informational Assistance**:
173
- Suitable for providing explanations, summaries, and research-based responses for students, educators, and general users.
174
-
175
- 3. **Conversational AI and Chatbots**:
176
- Ideal for building intelligent conversational agents that require contextual understanding and dynamic response generation.
177
-
178
- 4. **Multilingual Applications**:
179
- Supports global communication, translations, and multilingual content generation.
180
-
181
- 5. **Structured Data Processing**:
182
- Capable of analyzing and generating structured outputs, such as tables and JSON, useful for data science and automation.
183
-
184
- 6. **Long-Form Content Generation**:
185
- Can generate extended responses, including articles, reports, and guides, maintaining coherence over large text outputs.
186
-
187
- ## **Limitations**
188
- 1. **Hardware Requirements**:
189
- Requires high-memory GPUs or TPUs due to its large parameter size and long-context support.
190
-
191
- 2. **Potential Bias in Responses**:
192
- While designed to be neutral, outputs may still reflect biases present in training data.
193
-
194
- 3. **Inconsistent Outputs in Creative Tasks**:
195
- May produce variable results in storytelling and highly subjective topics.
196
-
197
- 4. **Limited Real-World Awareness**:
198
- Does not have access to real-time events beyond its training cutoff.
199
-
200
- 5. **Error Propagation in Extended Outputs**:
201
- Minor errors in early responses may affect overall coherence in long-form outputs.
202
-
203
- 6. **Prompt Sensitivity**:
204
- The effectiveness of responses may depend on how well the input prompt is structured.
205
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
206
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Epimetheus-14B-Axo-details)!
207
- Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FEpimetheus-14B-Axo&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
208
-
209
- | Metric |Value (%)|
210
- |-------------------|--------:|
211
- |**Average** | 39.08|
212
- |IFEval (0-Shot) | 55.46|
213
- |BBH (3-Shot) | 51.46|
214
- |MATH Lvl 5 (4-Shot)| 41.01|
215
- |GPQA (0-shot) | 19.02|
216
- |MuSR (0-shot) | 19.71|
217
- |MMLU-PRO (5-shot) | 47.82|
218
-
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - zho
5
+ - eng
6
+ - fra
7
+ - spa
8
+ - por
9
+ - deu
10
+ - ita
11
+ - rus
12
+ - jpn
13
+ - kor
14
+ - vie
15
+ - tha
16
+ - ara
17
+ base_model:
18
+ - Qwen/Qwen2.5-14B-Instruct
19
+ pipeline_tag: text-generation
20
+ library_name: transformers
21
+ tags:
22
+ - text-generation-inference
23
+ - math
24
+ - code
25
+ - trl
26
+ - sft
27
+ model-index:
28
+ - name: Epimetheus-14B-Axo
29
+ results:
30
+ - task:
31
+ type: text-generation
32
+ name: Text Generation
33
+ dataset:
34
+ name: IFEval (0-Shot)
35
+ type: wis-k/instruction-following-eval
36
+ split: train
37
+ args:
38
+ num_few_shot: 0
39
+ metrics:
40
+ - type: inst_level_strict_acc and prompt_level_strict_acc
41
+ value: 55.46
42
+ name: averaged accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: BBH (3-Shot)
51
+ type: SaylorTwift/bbh
52
+ split: test
53
+ args:
54
+ num_few_shot: 3
55
+ metrics:
56
+ - type: acc_norm
57
+ value: 51.46
58
+ name: normalized accuracy
59
+ source:
60
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
61
+ name: Open LLM Leaderboard
62
+ - task:
63
+ type: text-generation
64
+ name: Text Generation
65
+ dataset:
66
+ name: MATH Lvl 5 (4-Shot)
67
+ type: lighteval/MATH-Hard
68
+ split: test
69
+ args:
70
+ num_few_shot: 4
71
+ metrics:
72
+ - type: exact_match
73
+ value: 41.01
74
+ name: exact match
75
+ source:
76
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
77
+ name: Open LLM Leaderboard
78
+ - task:
79
+ type: text-generation
80
+ name: Text Generation
81
+ dataset:
82
+ name: GPQA (0-shot)
83
+ type: Idavidrein/gpqa
84
+ split: train
85
+ args:
86
+ num_few_shot: 0
87
+ metrics:
88
+ - type: acc_norm
89
+ value: 19.02
90
+ name: acc_norm
91
+ source:
92
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
93
+ name: Open LLM Leaderboard
94
+ - task:
95
+ type: text-generation
96
+ name: Text Generation
97
+ dataset:
98
+ name: MuSR (0-shot)
99
+ type: TAUR-Lab/MuSR
100
+ args:
101
+ num_few_shot: 0
102
+ metrics:
103
+ - type: acc_norm
104
+ value: 19.71
105
+ name: acc_norm
106
+ source:
107
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
108
+ name: Open LLM Leaderboard
109
+ - task:
110
+ type: text-generation
111
+ name: Text Generation
112
+ dataset:
113
+ name: MMLU-PRO (5-shot)
114
+ type: TIGER-Lab/MMLU-Pro
115
+ config: main
116
+ split: test
117
+ args:
118
+ num_few_shot: 5
119
+ metrics:
120
+ - type: acc
121
+ value: 47.82
122
+ name: accuracy
123
+ source:
124
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FEpimetheus-14B-Axo
125
+ name: Open LLM Leaderboard
126
+ ---
127
+ ![aszsdcazss.png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/jU62zQyKIuTfwXewv4T0Y.png)
128
+
129
+ # **Epimetheus-14B-Axo**
130
+
131
+ > Epimetheus-14B-Axo is based on the Qwen 2.5 14B modality architecture, designed to enhance the reasoning capabilities of 14B-parameter models. This model is optimized for general-purpose reasoning and answering, excelling in contextual understanding, logical deduction, and multi-step problem-solving. It has been fine-tuned using a long chain-of-thought reasoning model and specialized datasets to improve comprehension, structured responses, and conversational intelligence.
132
+
133
+ ## **Key Improvements**
134
+ 1. **Enhanced General Knowledge**: The model provides broad knowledge across various domains, improving capabilities in answering questions accurately and generating coherent responses.
135
+ 2. **Improved Instruction Following**: Significant advancements in understanding and following complex instructions, generating structured responses, and maintaining coherence over extended interactions.
136
+ 3. **Versatile Adaptability**: More resilient to diverse prompts, enhancing its ability to handle a wide range of topics and conversation styles, including open-ended and structured inquiries.
137
+ 4. **Long-Context Support**: Supports up to 128K tokens for input context and can generate up to 8K tokens in a single output, making it ideal for detailed responses.
138
+ 5. **Multilingual Proficiency**: Supports over 29 languages, including English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
139
+
140
+ ## **Quickstart with transformers**
141
+
142
+ Here is a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and generate content:
143
+
144
+ ```python
145
+ from transformers import AutoModelForCausalLM, AutoTokenizer
146
+
147
+ model_name = "prithivMLmods/Epimetheus-14B-Axo"
148
+
149
+ model = AutoModelForCausalLM.from_pretrained(
150
+ model_name,
151
+ torch_dtype="auto",
152
+ device_map="auto"
153
+ )
154
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
155
+
156
+ prompt = "What are the key principles of general-purpose AI?"
157
+ messages = [
158
+ {"role": "system", "content": "You are a helpful assistant capable of answering a wide range of questions."},
159
+ {"role": "user", "content": prompt}
160
+ ]
161
+ text = tokenizer.apply_chat_template(
162
+ messages,
163
+ tokenize=False,
164
+ add_generation_prompt=True
165
+ )
166
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
167
+
168
+ generated_ids = model.generate(
169
+ **model_inputs,
170
+ max_new_tokens=512
171
+ )
172
+ generated_ids = [
173
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
174
+ ]
175
+
176
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
177
+ ```
178
+
179
+ ## **Intended Use**
180
+ 1. **General-Purpose Reasoning**:
181
+ Designed for broad applicability, assisting with logical reasoning, answering diverse questions, and solving general knowledge problems.
182
+
183
+ 2. **Educational and Informational Assistance**:
184
+ Suitable for providing explanations, summaries, and research-based responses for students, educators, and general users.
185
+
186
+ 3. **Conversational AI and Chatbots**:
187
+ Ideal for building intelligent conversational agents that require contextual understanding and dynamic response generation.
188
+
189
+ 4. **Multilingual Applications**:
190
+ Supports global communication, translations, and multilingual content generation.
191
+
192
+ 5. **Structured Data Processing**:
193
+ Capable of analyzing and generating structured outputs, such as tables and JSON, useful for data science and automation.
194
+
195
+ 6. **Long-Form Content Generation**:
196
+ Can generate extended responses, including articles, reports, and guides, maintaining coherence over large text outputs.
197
+
198
+ ## **Limitations**
199
+ 1. **Hardware Requirements**:
200
+ Requires high-memory GPUs or TPUs due to its large parameter size and long-context support.
201
+
202
+ 2. **Potential Bias in Responses**:
203
+ While designed to be neutral, outputs may still reflect biases present in training data.
204
+
205
+ 3. **Inconsistent Outputs in Creative Tasks**:
206
+ May produce variable results in storytelling and highly subjective topics.
207
+
208
+ 4. **Limited Real-World Awareness**:
209
+ Does not have access to real-time events beyond its training cutoff.
210
+
211
+ 5. **Error Propagation in Extended Outputs**:
212
+ Minor errors in early responses may affect overall coherence in long-form outputs.
213
+
214
+ 6. **Prompt Sensitivity**:
215
+ The effectiveness of responses may depend on how well the input prompt is structured.
216
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
217
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Epimetheus-14B-Axo-details)!
218
+ Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FEpimetheus-14B-Axo&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
219
+
220
+ | Metric |Value (%)|
221
+ |-------------------|--------:|
222
+ |**Average** | 39.08|
223
+ |IFEval (0-Shot) | 55.46|
224
+ |BBH (3-Shot) | 51.46|
225
+ |MATH Lvl 5 (4-Shot)| 41.01|
226
+ |GPQA (0-shot) | 19.02|
227
+ |MuSR (0-shot) | 19.71|
228
+ |MMLU-PRO (5-shot) | 47.82|
229
+