Hugging Face's logo

qingy2024
/
Qwen2.5-Ultimate-14B-Instruct

Safetensors
qwen2
Model card Files
xet
Community
3
qingy2024 lbourdois commited on
Commit
69c7549
·
verified ·
1 Parent(s): 62327b9

Improve language tag (#2)

Browse files

- Improve language tag (6a44b62f0d9badf894c0716b3e6911f19f5440f5)


Co-authored-by: Loïck BOURDOIS <[email protected]>

Files changed (1) hide show
  1. README.md +173 -159
README.md CHANGED
@@ -1,159 +1,173 @@
1
- ---
2
- library_name: transformers
3
- tags:
4
- - mergekit
5
- - merge
6
- base_model:
7
- - Qwen/Qwen2.5-14B-Instruct
8
- - qingy2019/Qwen2.5-Math-14B-Instruct
9
- - Qwen/Qwen2.5-14B
10
- model-index:
11
- - name: Qwen2.5-Ultimate-14B-Instruct
12
- results:
13
- - task:
14
- type: text-generation
15
- name: Text Generation
16
- dataset:
17
- name: IFEval (0-Shot)
18
- type: HuggingFaceH4/ifeval
19
- args:
20
- num_few_shot: 0
21
- metrics:
22
- - type: inst_level_strict_acc and prompt_level_strict_acc
23
- value: 39.38
24
- name: strict accuracy
25
- source:
26
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
27
- name: Open LLM Leaderboard
28
- - task:
29
- type: text-generation
30
- name: Text Generation
31
- dataset:
32
- name: BBH (3-Shot)
33
- type: BBH
34
- args:
35
- num_few_shot: 3
36
- metrics:
37
- - type: acc_norm
38
- value: 40.58
39
- name: normalized accuracy
40
- source:
41
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
42
- name: Open LLM Leaderboard
43
- - task:
44
- type: text-generation
45
- name: Text Generation
46
- dataset:
47
- name: MATH Lvl 5 (4-Shot)
48
- type: hendrycks/competition_math
49
- args:
50
- num_few_shot: 4
51
- metrics:
52
- - type: exact_match
53
- value: 28.02
54
- name: exact match
55
- source:
56
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
57
- name: Open LLM Leaderboard
58
- - task:
59
- type: text-generation
60
- name: Text Generation
61
- dataset:
62
- name: GPQA (0-shot)
63
- type: Idavidrein/gpqa
64
- args:
65
- num_few_shot: 0
66
- metrics:
67
- - type: acc_norm
68
- value: 14.21
69
- name: acc_norm
70
- source:
71
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
72
- name: Open LLM Leaderboard
73
- - task:
74
- type: text-generation
75
- name: Text Generation
76
- dataset:
77
- name: MuSR (0-shot)
78
- type: TAUR-Lab/MuSR
79
- args:
80
- num_few_shot: 0
81
- metrics:
82
- - type: acc_norm
83
- value: 9.89
84
- name: acc_norm
85
- source:
86
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
87
- name: Open LLM Leaderboard
88
- - task:
89
- type: text-generation
90
- name: Text Generation
91
- dataset:
92
- name: MMLU-PRO (5-shot)
93
- type: TIGER-Lab/MMLU-Pro
94
- config: main
95
- split: test
96
- args:
97
- num_few_shot: 5
98
- metrics:
99
- - type: acc
100
- value: 43.66
101
- name: accuracy
102
- source:
103
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
104
- name: Open LLM Leaderboard
105
- ---
106
- # Qwen2.5 Ultimate 14B Instruct
107
-
108
- Merged using rombodawg's method and using the first iteration of my Qwen2.5 Math 14B Instruct.
109
-
110
- ## Merge Details
111
- ### Merge Method
112
-
113
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
114
-
115
- ### Models Merged
116
-
117
- The following models were included in the merge:
118
- * [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
119
- * [qingy2019/Qwen2.5-Math-14B-Instruct](https://huggingface.co/qingy2019/Qwen2.5-Math-14B-Instruct)
120
-
121
- ### Configuration
122
-
123
- The following YAML configuration was used to produce this model:
124
-
125
- ```yaml
126
- models:
127
- - model: qingy2019/Qwen2.5-Math-14B-Instruct
128
- parameters:
129
- weight: 1
130
- density: 1
131
- - model: Qwen/Qwen2.5-14B-Instruct
132
- parameters:
133
- weight: 1
134
- density: 1
135
- merge_method: ties
136
- base_model: Qwen/Qwen2.5-14B
137
- parameters:
138
- weight: 1
139
- density: 1
140
- normalize: true
141
- int8_mask: true
142
- tokenizer_source: qingy2019/Qwen2.5-Math-14B-Instruct
143
- dtype: bfloat16
144
-
145
- ```
146
-
147
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
148
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_qingy2019__Qwen2.5-Ultimate-14B-Instruct)
149
-
150
- | Metric |Value|
151
- |-------------------|----:|
152
- |Avg. |29.29|
153
- |IFEval (0-Shot) |39.38|
154
- |BBH (3-Shot) |40.58|
155
- |MATH Lvl 5 (4-Shot)|28.02|
156
- |GPQA (0-shot) |14.21|
157
- |MuSR (0-shot) | 9.89|
158
- |MMLU-PRO (5-shot) |43.66|
159
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - mergekit
5
+ - merge
6
+ base_model:
7
+ - Qwen/Qwen2.5-14B-Instruct
8
+ - qingy2019/Qwen2.5-Math-14B-Instruct
9
+ - Qwen/Qwen2.5-14B
10
+ language:
11
+ - zho
12
+ - eng
13
+ - fra
14
+ - spa
15
+ - por
16
+ - deu
17
+ - ita
18
+ - rus
19
+ - jpn
20
+ - kor
21
+ - vie
22
+ - tha
23
+ - ara
24
+ model-index:
25
+ - name: Qwen2.5-Ultimate-14B-Instruct
26
+ results:
27
+ - task:
28
+ type: text-generation
29
+ name: Text Generation
30
+ dataset:
31
+ name: IFEval (0-Shot)
32
+ type: HuggingFaceH4/ifeval
33
+ args:
34
+ num_few_shot: 0
35
+ metrics:
36
+ - type: inst_level_strict_acc and prompt_level_strict_acc
37
+ value: 39.38
38
+ name: strict accuracy
39
+ source:
40
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
41
+ name: Open LLM Leaderboard
42
+ - task:
43
+ type: text-generation
44
+ name: Text Generation
45
+ dataset:
46
+ name: BBH (3-Shot)
47
+ type: BBH
48
+ args:
49
+ num_few_shot: 3
50
+ metrics:
51
+ - type: acc_norm
52
+ value: 40.58
53
+ name: normalized accuracy
54
+ source:
55
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
56
+ name: Open LLM Leaderboard
57
+ - task:
58
+ type: text-generation
59
+ name: Text Generation
60
+ dataset:
61
+ name: MATH Lvl 5 (4-Shot)
62
+ type: hendrycks/competition_math
63
+ args:
64
+ num_few_shot: 4
65
+ metrics:
66
+ - type: exact_match
67
+ value: 28.02
68
+ name: exact match
69
+ source:
70
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
71
+ name: Open LLM Leaderboard
72
+ - task:
73
+ type: text-generation
74
+ name: Text Generation
75
+ dataset:
76
+ name: GPQA (0-shot)
77
+ type: Idavidrein/gpqa
78
+ args:
79
+ num_few_shot: 0
80
+ metrics:
81
+ - type: acc_norm
82
+ value: 14.21
83
+ name: acc_norm
84
+ source:
85
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
86
+ name: Open LLM Leaderboard
87
+ - task:
88
+ type: text-generation
89
+ name: Text Generation
90
+ dataset:
91
+ name: MuSR (0-shot)
92
+ type: TAUR-Lab/MuSR
93
+ args:
94
+ num_few_shot: 0
95
+ metrics:
96
+ - type: acc_norm
97
+ value: 9.89
98
+ name: acc_norm
99
+ source:
100
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
101
+ name: Open LLM Leaderboard
102
+ - task:
103
+ type: text-generation
104
+ name: Text Generation
105
+ dataset:
106
+ name: MMLU-PRO (5-shot)
107
+ type: TIGER-Lab/MMLU-Pro
108
+ config: main
109
+ split: test
110
+ args:
111
+ num_few_shot: 5
112
+ metrics:
113
+ - type: acc
114
+ value: 43.66
115
+ name: accuracy
116
+ source:
117
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=qingy2019/Qwen2.5-Ultimate-14B-Instruct
118
+ name: Open LLM Leaderboard
119
+ ---
120
+ # Qwen2.5 Ultimate 14B Instruct
121
+
122
+ Merged using rombodawg's method and using the first iteration of my Qwen2.5 Math 14B Instruct.
123
+
124
+ ## Merge Details
125
+ ### Merge Method
126
+
127
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
128
+
129
+ ### Models Merged
130
+
131
+ The following models were included in the merge:
132
+ * [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
133
+ * [qingy2019/Qwen2.5-Math-14B-Instruct](https://huggingface.co/qingy2019/Qwen2.5-Math-14B-Instruct)
134
+
135
+ ### Configuration
136
+
137
+ The following YAML configuration was used to produce this model:
138
+
139
+ ```yaml
140
+ models:
141
+ - model: qingy2019/Qwen2.5-Math-14B-Instruct
142
+ parameters:
143
+ weight: 1
144
+ density: 1
145
+ - model: Qwen/Qwen2.5-14B-Instruct
146
+ parameters:
147
+ weight: 1
148
+ density: 1
149
+ merge_method: ties
150
+ base_model: Qwen/Qwen2.5-14B
151
+ parameters:
152
+ weight: 1
153
+ density: 1
154
+ normalize: true
155
+ int8_mask: true
156
+ tokenizer_source: qingy2019/Qwen2.5-Math-14B-Instruct
157
+ dtype: bfloat16
158
+
159
+ ```
160
+
161
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
162
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_qingy2019__Qwen2.5-Ultimate-14B-Instruct)
163
+
164
+ | Metric |Value|
165
+ |-------------------|----:|
166
+ |Avg. |29.29|
167
+ |IFEval (0-Shot) |39.38|
168
+ |BBH (3-Shot) |40.58|
169
+ |MATH Lvl 5 (4-Shot)|28.02|
170
+ |GPQA (0-shot) |14.21|
171
+ |MuSR (0-shot) | 9.89|
172
+ |MMLU-PRO (5-shot) |43.66|
173
+