Hugging Face's logo

tensorblock
/
Qwen2.5-Lumen-14B-GGUF

Text Generation
Transformers
GGUF
English
qwen
qwen2.5
finetune
dpo
orpo
qwen2
chat
conversational
instruct
storywriting
roleplay
novelwriting
TensorBlock
GGUF
Eval Results
Inference Endpoints
Model card Files Community
morriszms commited on
Commit
e8dea49
•
1 Parent(s): bcbf55a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -12
README.md CHANGED
@@ -138,8 +138,16 @@ This repo contains GGUF format model files for [v000000/Qwen2.5-Lumen-14B](https
138
 
139
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
140
 
 
 
 
 
 
 
 
141
  ## Prompt template
142
 
 
143
  ```
144
  <|im_start|>system
145
  {system_prompt}<|im_end|>
@@ -152,18 +160,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
152
 
153
  | Filename | Quant type | File Size | Description |
154
  | -------- | ---------- | --------- | ----------- |
155
- | [Qwen2.5-Lumen-14B-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q2_K.gguf) | Q2_K | 5.374 GB | smallest, significant quality loss - not recommended for most purposes |
156
- | [Qwen2.5-Lumen-14B-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q3_K_S.gguf) | Q3_K_S | 6.202 GB | very small, high quality loss |
157
- | [Qwen2.5-Lumen-14B-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q3_K_M.gguf) | Q3_K_M | 6.835 GB | very small, high quality loss |
158
- | [Qwen2.5-Lumen-14B-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q3_K_L.gguf) | Q3_K_L | 7.381 GB | small, substantial quality loss |
159
- | [Qwen2.5-Lumen-14B-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q4_0.gguf) | Q4_0 | 7.933 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
160
- | [Qwen2.5-Lumen-14B-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q4_K_S.gguf) | Q4_K_S | 7.985 GB | small, greater quality loss |
161
- | [Qwen2.5-Lumen-14B-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q4_K_M.gguf) | Q4_K_M | 8.371 GB | medium, balanced quality - recommended |
162
- | [Qwen2.5-Lumen-14B-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q5_0.gguf) | Q5_0 | 9.561 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
163
- | [Qwen2.5-Lumen-14B-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q5_K_S.gguf) | Q5_K_S | 9.561 GB | large, low quality loss - recommended |
164
- | [Qwen2.5-Lumen-14B-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q5_K_M.gguf) | Q5_K_M | 9.787 GB | large, very low quality loss - recommended |
165
- | [Qwen2.5-Lumen-14B-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q6_K.gguf) | Q6_K | 11.292 GB | very large, extremely low quality loss |
166
- | [Qwen2.5-Lumen-14B-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/tree/main/Qwen2.5-Lumen-14B-Q8_0.gguf) | Q8_0 | 14.623 GB | very large, extremely low quality loss - not recommended |
167
 
168
 
169
  ## Downloading instruction
 
138
 
139
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
140
 
141
+
142
+ <div style="text-align: left; margin: 20px 0;">
143
+ <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
144
+ Run them on the TensorBlock client using your local machine ↗
145
+ </a>
146
+ </div>
147
+
148
  ## Prompt template
149
 
150
+
151
  ```
152
  <|im_start|>system
153
  {system_prompt}<|im_end|>
 
160
 
161
  | Filename | Quant type | File Size | Description |
162
  | -------- | ---------- | --------- | ----------- |
163
+ | [Qwen2.5-Lumen-14B-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q2_K.gguf) | Q2_K | 5.374 GB | smallest, significant quality loss - not recommended for most purposes |
164
+ | [Qwen2.5-Lumen-14B-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q3_K_S.gguf) | Q3_K_S | 6.202 GB | very small, high quality loss |
165
+ | [Qwen2.5-Lumen-14B-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q3_K_M.gguf) | Q3_K_M | 6.835 GB | very small, high quality loss |
166
+ | [Qwen2.5-Lumen-14B-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q3_K_L.gguf) | Q3_K_L | 7.381 GB | small, substantial quality loss |
167
+ | [Qwen2.5-Lumen-14B-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q4_0.gguf) | Q4_0 | 7.933 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
168
+ | [Qwen2.5-Lumen-14B-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q4_K_S.gguf) | Q4_K_S | 7.985 GB | small, greater quality loss |
169
+ | [Qwen2.5-Lumen-14B-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q4_K_M.gguf) | Q4_K_M | 8.371 GB | medium, balanced quality - recommended |
170
+ | [Qwen2.5-Lumen-14B-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q5_0.gguf) | Q5_0 | 9.561 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
171
+ | [Qwen2.5-Lumen-14B-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q5_K_S.gguf) | Q5_K_S | 9.561 GB | large, low quality loss - recommended |
172
+ | [Qwen2.5-Lumen-14B-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q5_K_M.gguf) | Q5_K_M | 9.787 GB | large, very low quality loss - recommended |
173
+ | [Qwen2.5-Lumen-14B-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q6_K.gguf) | Q6_K | 11.292 GB | very large, extremely low quality loss |
174
+ | [Qwen2.5-Lumen-14B-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Lumen-14B-GGUF/blob/main/Qwen2.5-Lumen-14B-Q8_0.gguf) | Q8_0 | 14.623 GB | very large, extremely low quality loss - not recommended |
175
 
176
 
177
  ## Downloading instruction