Hugging Face's logo

GPT4All-Community
/
Yi-1.5-9B-Chat-16K-GGUF

Text Generation
Transformers
GGUF
English
Chinese
text-generation-inference
GGUF
GPT4All
conversational
Model card Files Community
ThiloteE commited on
Commit
21e5c7a
·
verified ·
1 Parent(s): ae9a0b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +194 -3
README.md CHANGED
@@ -1,3 +1,194 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: 01-ai/Yi-1.5-9B-Chat-16K
3
+ pipeline_tag: text-generation
4
+ inference: false
5
+ model_creator: 01-AI
6
+ model_name: Yi-1.5-9B-Chat-16K
7
+ model_type: llama
8
+ language:
9
+ - en
10
+ - zh
11
+ library_name: transformers
12
+ license: apache-2.0
13
+ quantized_by: ThiloteE
14
+ tags:
15
+ - text-generation-inference
16
+ - transformers
17
+ - GGUF
18
+ - GPT4All
19
+
20
+ ---
21
+ # About
22
+
23
+ <!-- ### quantize_version: 2 -->
24
+ <!-- ### output_tensor_quantised: 1 -->
25
+ <!-- ### convert_type: hf -->
26
+ <!-- ### vocab_type: -->
27
+ <!-- ### tags: -->
28
+
29
+ - Static quants of https://huggingface.co/01-ai/Yi-1.5-9B-Chat-16K
30
+ - Quantized with llama.cpp commit [c3776ca](https://github.com/ggerganov/llama.cpp/commit/c3776cacabce2ee35f172fb72be7a519752125fa) by [ThiloteE](https://huggingface.co/ThiloteE)
31
+
32
+ # Notes
33
+
34
+ These quants were created with a customized configuration that have been proven to not cause visible end of string (eos) tokens during inference with [GPT4All](https://www.nomic.ai/gpt4all).
35
+ The config.json, generation_config.json and tokenizer_config.json differ from the original configuration as can be found in the original model's repository at the time of creation of these quants.
36
+
37
+ # Prompt Template (for GPT4All)
38
+
39
+ System Prompt:
40
+ ```
41
+ <|im_start|>system
42
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.<|im_end|>
43
+
44
+ ```
45
+
46
+
47
+ Chat Template:
48
+ ```
49
+ <|im_start|>user
50
+ %1<|im_end|>
51
+ <|im_start|>assistant
52
+ %2<|im_end|>
53
+
54
+ ```
55
+
56
+ Do not miss the newlines at the end!
57
+
58
+ # Context Length
59
+
60
+ `16384`
61
+
62
+
63
+ # Provided Quants
64
+
65
+
66
+ | Link | Type | Size/GB | Notes |
67
+ |:-----|:-----|--------:|:------|
68
+ | [GGUF](https://huggingface.co/mradermacher/pstella-16b-GGUF/resolve/main/pstella-16b.Q4_K_M.gguf) | Q4_0 | 4.9 | fast, recommended |
69
+ | [GGUF](https://huggingface.co/mradermacher/pstella-16b-GGUF/resolve/main/pstella-16b.f16.gguf) | f16 | 17.2 | 16 bpw, overkill |
70
+
71
+
72
+
73
+
74
+ # About GGUF
75
+
76
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
77
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
78
+ more details, including on how to concatenate multi-part files.
79
+
80
+ Here is a handy graph by ikawrakow comparing some quant types (lower is better):
81
+
82
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
83
+
84
+ And here are Artefact2's thoughts on the matter:
85
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
86
+
87
+ # Thanks
88
+
89
+ I thank Mradermacher and TheBloke for Inspiration to this model card and their contributions to open source. I thank 3Simplex for everything.
90
+ Shoutout to the GPT4All and llama.cpp communities :-)
91
+
92
+
93
+ ------
94
+
95
+ <!-- footer end -->
96
+ <!-- original-model-card start -->
97
+ # Original Model card:
98
+
99
+ ---
100
+ license: apache-2.0
101
+ ---
102
+ <div align="center">
103
+
104
+ <picture>
105
+ <img src="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_light.svg" width="150px">
106
+ </picture>
107
+
108
+ </div>
109
+
110
+ <p align="center">
111
+ <a href="https://github.com/01-ai">🐙 GitHub</a> •
112
+ <a href="https://discord.gg/hYUwWddeAu">👾 Discord</a> •
113
+ <a href="https://twitter.com/01ai_yi">🐤 Twitter</a> •
114
+ <a href="https://github.com/01-ai/Yi-1.5/issues/2">💬 WeChat</a>
115
+ <br/>
116
+ <a href="https://arxiv.org/abs/2403.04652">📝 Paper</a> •
117
+ <a href="https://01-ai.github.io/">💪 Tech Blog</a> •
118
+ <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">🙌 FAQ</a> •
119
+ <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">📗 Learning Hub</a>
120
+ </p>
121
+
122
+ # Intro
123
+
124
+ Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples.
125
+
126
+ Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension.
127
+
128
+ <div align="center">
129
+
130
+ Model | Context Length | Pre-trained Tokens
131
+ | :------------: | :------------: | :------------: |
132
+ | Yi-1.5 | 4K, 16K, 32K | 3.6T
133
+
134
+ </div>
135
+
136
+ # Models
137
+
138
+ - Chat models
139
+
140
+ <div align="center">
141
+
142
+ | Name | Download |
143
+ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
144
+ | Yi-1.5-34B-Chat | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI)|
145
+ | Yi-1.5-34B-Chat-16K | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
146
+ | Yi-1.5-9B-Chat | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
147
+ | Yi-1.5-9B-Chat-16K | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
148
+ | Yi-1.5-6B-Chat | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
149
+
150
+ </div>
151
+
152
+ - Base models
153
+
154
+ <div align="center">
155
+
156
+ | Name | Download |
157
+ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
158
+ | Yi-1.5-34B | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
159
+ | Yi-1.5-34B-32K | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
160
+ | Yi-1.5-9B | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
161
+ | Yi-1.5-9B-32K | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
162
+ | Yi-1.5-6B | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
163
+
164
+ </div>
165
+
166
+ # Benchmarks
167
+
168
+ - Chat models
169
+
170
+ Yi-1.5-34B-Chat is on par with or excels beyond larger models in most benchmarks.
171
+
172
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/KcsJ9Oc1VnEmfCDEJc5cd.png)
173
+
174
+ Yi-1.5-9B-Chat is the top performer among similarly sized open-source models.
175
+
176
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/xf6pLg5jqRCwjlh6m3t6_.png)
177
+
178
+ - Base models
179
+
180
+ Yi-1.5-34B is on par with or excels beyond larger models in some benchmarks.
181
+
182
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/BwU7QM-03dZvZzwdIE1xY.png)
183
+
184
+ Yi-1.5-9B is the top performer among similarly sized open-source models.
185
+
186
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/y-EYSYPT-3aWLJ0x8R94F.png)
187
+
188
+ # Quick Start
189
+
190
+ For getting up and running with Yi-1.5 models quickly, see [README](https://github.com/01-ai/Yi-1.5).
191
+
192
+
193
+ <!-- original-model-card end -->
194
+ <!-- end -->