lbourdois commited on
Commit
dfb7602
·
verified ·
1 Parent(s): 4d6aadc

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +153 -139
README.md CHANGED
@@ -1,139 +1,153 @@
1
- ---
2
- license: apache-2.0
3
- base_model:
4
- - Qwen/Qwen2.5-14B
5
- model-index:
6
- - name: Virtuoso-Small
7
- results:
8
- - task:
9
- type: text-generation
10
- name: Text Generation
11
- dataset:
12
- name: IFEval (0-Shot)
13
- type: HuggingFaceH4/ifeval
14
- args:
15
- num_few_shot: 0
16
- metrics:
17
- - type: inst_level_strict_acc and prompt_level_strict_acc
18
- value: 79.35
19
- name: strict accuracy
20
- source:
21
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
22
- name: Open LLM Leaderboard
23
- - task:
24
- type: text-generation
25
- name: Text Generation
26
- dataset:
27
- name: BBH (3-Shot)
28
- type: BBH
29
- args:
30
- num_few_shot: 3
31
- metrics:
32
- - type: acc_norm
33
- value: 50.4
34
- name: normalized accuracy
35
- source:
36
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
37
- name: Open LLM Leaderboard
38
- - task:
39
- type: text-generation
40
- name: Text Generation
41
- dataset:
42
- name: MATH Lvl 5 (4-Shot)
43
- type: hendrycks/competition_math
44
- args:
45
- num_few_shot: 4
46
- metrics:
47
- - type: exact_match
48
- value: 34.29
49
- name: exact match
50
- source:
51
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
52
- name: Open LLM Leaderboard
53
- - task:
54
- type: text-generation
55
- name: Text Generation
56
- dataset:
57
- name: GPQA (0-shot)
58
- type: Idavidrein/gpqa
59
- args:
60
- num_few_shot: 0
61
- metrics:
62
- - type: acc_norm
63
- value: 11.52
64
- name: acc_norm
65
- source:
66
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
67
- name: Open LLM Leaderboard
68
- - task:
69
- type: text-generation
70
- name: Text Generation
71
- dataset:
72
- name: MuSR (0-shot)
73
- type: TAUR-Lab/MuSR
74
- args:
75
- num_few_shot: 0
76
- metrics:
77
- - type: acc_norm
78
- value: 14.44
79
- name: acc_norm
80
- source:
81
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
82
- name: Open LLM Leaderboard
83
- - task:
84
- type: text-generation
85
- name: Text Generation
86
- dataset:
87
- name: MMLU-PRO (5-shot)
88
- type: TIGER-Lab/MMLU-Pro
89
- config: main
90
- split: test
91
- args:
92
- num_few_shot: 5
93
- metrics:
94
- - type: acc
95
- value: 46.57
96
- name: accuracy
97
- source:
98
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
99
- name: Open LLM Leaderboard
100
- ---
101
-
102
- <!---
103
- <div align="center">
104
- <img src="https://i.ibb.co/pXD6Bcv/SW2-U-g-QQLSH1-ZAbxhs-Iu-A.webp" alt="Virtuoso-Small" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
105
- </div>
106
- --->
107
-
108
- GGUF Available [Here](https://huggingface.co/arcee-ai/Virtuoso-Small-GGUF)
109
-
110
- # Virtuoso-Small
111
-
112
- Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Medium and Virtuoso-Large, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).
113
-
114
- ## Key Features
115
-
116
- - **Compact and Efficient**: With 14 billion parameters, Virtuoso-Small provides a high-performance solution optimized for smaller hardware configurations without sacrificing quality.
117
- - **Business-Oriented**: Tailored for use cases such as customer support, content creation, and technical assistance, Virtuoso-Small meets the demands of modern enterprises.
118
- - **Scalable Ecosystem**: Part of the Virtuoso series, Virtuoso-Small is fully interoperable with its larger siblings, Medium and Large, enabling seamless scaling as your needs grow.
119
-
120
- ---
121
-
122
- ## Deployment Options
123
-
124
- Virtuoso-Small is available under the Apache-2.0 license and can be deployed locally or accessed through an API at [models.arcee.ai](https://models.arcee.ai). For larger-scale or more demanding applications, consider Virtuoso-Medium or Virtuoso-Large.
125
-
126
-
127
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
128
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Virtuoso-Small)
129
-
130
- | Metric |Value|
131
- |-------------------|----:|
132
- |Avg. |39.43|
133
- |IFEval (0-Shot) |79.35|
134
- |BBH (3-Shot) |50.40|
135
- |MATH Lvl 5 (4-Shot)|34.29|
136
- |GPQA (0-shot) |11.52|
137
- |MuSR (0-shot) |14.44|
138
- |MMLU-PRO (5-shot) |46.57|
139
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen2.5-14B
5
+ language:
6
+ - zho
7
+ - eng
8
+ - fra
9
+ - spa
10
+ - por
11
+ - deu
12
+ - ita
13
+ - rus
14
+ - jpn
15
+ - kor
16
+ - vie
17
+ - tha
18
+ - ara
19
+ model-index:
20
+ - name: Virtuoso-Small
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ name: Text Generation
25
+ dataset:
26
+ name: IFEval (0-Shot)
27
+ type: HuggingFaceH4/ifeval
28
+ args:
29
+ num_few_shot: 0
30
+ metrics:
31
+ - type: inst_level_strict_acc and prompt_level_strict_acc
32
+ value: 79.35
33
+ name: strict accuracy
34
+ source:
35
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
36
+ name: Open LLM Leaderboard
37
+ - task:
38
+ type: text-generation
39
+ name: Text Generation
40
+ dataset:
41
+ name: BBH (3-Shot)
42
+ type: BBH
43
+ args:
44
+ num_few_shot: 3
45
+ metrics:
46
+ - type: acc_norm
47
+ value: 50.4
48
+ name: normalized accuracy
49
+ source:
50
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
51
+ name: Open LLM Leaderboard
52
+ - task:
53
+ type: text-generation
54
+ name: Text Generation
55
+ dataset:
56
+ name: MATH Lvl 5 (4-Shot)
57
+ type: hendrycks/competition_math
58
+ args:
59
+ num_few_shot: 4
60
+ metrics:
61
+ - type: exact_match
62
+ value: 34.29
63
+ name: exact match
64
+ source:
65
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
66
+ name: Open LLM Leaderboard
67
+ - task:
68
+ type: text-generation
69
+ name: Text Generation
70
+ dataset:
71
+ name: GPQA (0-shot)
72
+ type: Idavidrein/gpqa
73
+ args:
74
+ num_few_shot: 0
75
+ metrics:
76
+ - type: acc_norm
77
+ value: 11.52
78
+ name: acc_norm
79
+ source:
80
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
81
+ name: Open LLM Leaderboard
82
+ - task:
83
+ type: text-generation
84
+ name: Text Generation
85
+ dataset:
86
+ name: MuSR (0-shot)
87
+ type: TAUR-Lab/MuSR
88
+ args:
89
+ num_few_shot: 0
90
+ metrics:
91
+ - type: acc_norm
92
+ value: 14.44
93
+ name: acc_norm
94
+ source:
95
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
96
+ name: Open LLM Leaderboard
97
+ - task:
98
+ type: text-generation
99
+ name: Text Generation
100
+ dataset:
101
+ name: MMLU-PRO (5-shot)
102
+ type: TIGER-Lab/MMLU-Pro
103
+ config: main
104
+ split: test
105
+ args:
106
+ num_few_shot: 5
107
+ metrics:
108
+ - type: acc
109
+ value: 46.57
110
+ name: accuracy
111
+ source:
112
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=arcee-ai/Virtuoso-Small
113
+ name: Open LLM Leaderboard
114
+ ---
115
+
116
+ <!---
117
+ <div align="center">
118
+ <img src="https://i.ibb.co/pXD6Bcv/SW2-U-g-QQLSH1-ZAbxhs-Iu-A.webp" alt="Virtuoso-Small" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
119
+ </div>
120
+ --->
121
+
122
+ GGUF Available [Here](https://huggingface.co/arcee-ai/Virtuoso-Small-GGUF)
123
+
124
+ # Virtuoso-Small
125
+
126
+ Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Medium and Virtuoso-Large, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).
127
+
128
+ ## Key Features
129
+
130
+ - **Compact and Efficient**: With 14 billion parameters, Virtuoso-Small provides a high-performance solution optimized for smaller hardware configurations without sacrificing quality.
131
+ - **Business-Oriented**: Tailored for use cases such as customer support, content creation, and technical assistance, Virtuoso-Small meets the demands of modern enterprises.
132
+ - **Scalable Ecosystem**: Part of the Virtuoso series, Virtuoso-Small is fully interoperable with its larger siblings, Medium and Large, enabling seamless scaling as your needs grow.
133
+
134
+ ---
135
+
136
+ ## Deployment Options
137
+
138
+ Virtuoso-Small is available under the Apache-2.0 license and can be deployed locally or accessed through an API at [models.arcee.ai](https://models.arcee.ai). For larger-scale or more demanding applications, consider Virtuoso-Medium or Virtuoso-Large.
139
+
140
+
141
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
142
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Virtuoso-Small)
143
+
144
+ | Metric |Value|
145
+ |-------------------|----:|
146
+ |Avg. |39.43|
147
+ |IFEval (0-Shot) |79.35|
148
+ |BBH (3-Shot) |50.40|
149
+ |MATH Lvl 5 (4-Shot)|34.29|
150
+ |GPQA (0-shot) |11.52|
151
+ |MuSR (0-shot) |14.44|
152
+ |MMLU-PRO (5-shot) |46.57|
153
+