qingy2024 lbourdois commited on
Commit
c39ad7e
·
verified ·
1 Parent(s): b67c8a7

Improve language tag (#1)

Browse files

- Improve language tag (0f49ab35e8de0622e2b45d731892527a9cb4d34a)


Co-authored-by: Loïck BOURDOIS <[email protected]>

Files changed (1) hide show
  1. README.md +62 -49
README.md CHANGED
@@ -1,49 +1,62 @@
1
- ---
2
- base_model:
3
- - qingy2024/Qwen2.5-Math-14B-Instruct-Preview
4
- - arcee-ai/Virtuoso-Small
5
- - Qwen/Qwen2.5-14B
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # merge
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [qingy2024/Qwen2.5-Math-14B-Instruct-Preview](https://huggingface.co/qingy2024/Qwen2.5-Math-14B-Instruct-Preview)
25
- * [arcee-ai/Virtuoso-Small](https://huggingface.co/arcee-ai/Virtuoso-Small)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- models:
33
- - model: qingy2024/Qwen2.5-Math-14B-Instruct-Preview
34
- parameters:
35
- weight: 0.35
36
- density: 0.6
37
- - model: arcee-ai/Virtuoso-Small
38
- parameters:
39
- weight: 0.65
40
- density: 0.8
41
- base_model: Qwen/Qwen2.5-14B
42
- merge_method: dare_ties
43
- parameters:
44
- normalize: true
45
- int8_mask: true
46
- dtype: bfloat16
47
- tokenizer_source: Qwen/Qwen2.5-14B-Instruct
48
-
49
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - qingy2024/Qwen2.5-Math-14B-Instruct-Preview
4
+ - arcee-ai/Virtuoso-Small
5
+ - Qwen/Qwen2.5-14B
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+ language:
11
+ - zho
12
+ - eng
13
+ - fra
14
+ - spa
15
+ - por
16
+ - deu
17
+ - ita
18
+ - rus
19
+ - jpn
20
+ - kor
21
+ - vie
22
+ - tha
23
+ - ara
24
+ ---
25
+ # merge
26
+
27
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
28
+
29
+ ## Merge Details
30
+ ### Merge Method
31
+
32
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
33
+
34
+ ### Models Merged
35
+
36
+ The following models were included in the merge:
37
+ * [qingy2024/Qwen2.5-Math-14B-Instruct-Preview](https://huggingface.co/qingy2024/Qwen2.5-Math-14B-Instruct-Preview)
38
+ * [arcee-ai/Virtuoso-Small](https://huggingface.co/arcee-ai/Virtuoso-Small)
39
+
40
+ ### Configuration
41
+
42
+ The following YAML configuration was used to produce this model:
43
+
44
+ ```yaml
45
+ models:
46
+ - model: qingy2024/Qwen2.5-Math-14B-Instruct-Preview
47
+ parameters:
48
+ weight: 0.35
49
+ density: 0.6
50
+ - model: arcee-ai/Virtuoso-Small
51
+ parameters:
52
+ weight: 0.65
53
+ density: 0.8
54
+ base_model: Qwen/Qwen2.5-14B
55
+ merge_method: dare_ties
56
+ parameters:
57
+ normalize: true
58
+ int8_mask: true
59
+ dtype: bfloat16
60
+ tokenizer_source: Qwen/Qwen2.5-14B-Instruct
61
+
62
+ ```