Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -1,7 +1,6 @@
|
|
1 |
---
|
2 |
quantized_by: bartowski
|
3 |
pipeline_tag: text-generation
|
4 |
-
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
|
5 |
---
|
6 |
|
7 |
## Llamacpp imatrix Quantizations of Qwen2.5-Coder-14B-Instruct
|
@@ -36,7 +35,7 @@ Run them in [LM Studio](https://lmstudio.ai/)
|
|
36 |
| [Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf) | Q5_K_M | 10.51GB | false | High quality, *recommended*. |
|
37 |
| [Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf) | Q5_K_S | 10.27GB | false | High quality, *recommended*. |
|
38 |
| [Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf) | Q4_K_L | 9.57GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
39 |
-
| [Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf) | Q4_K_M | 8.99GB | false | Good quality, default size for
|
40 |
| [Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf) | Q3_K_XL | 8.61GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
41 |
| [Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf) | Q4_K_S | 8.57GB | false | Slightly lower quality with more space savings, *recommended*. |
|
42 |
| [Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf) | IQ4_NL | 8.55GB | false | Similar to IQ4_XS, but slightly larger. |
|
@@ -120,8 +119,8 @@ The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have
|
|
120 |
|
121 |
## Credits
|
122 |
|
123 |
-
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
|
124 |
|
125 |
-
Thank you ZeroWw for the inspiration to experiment with embed/output
|
126 |
|
127 |
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
|
|
|
1 |
---
|
2 |
quantized_by: bartowski
|
3 |
pipeline_tag: text-generation
|
|
|
4 |
---
|
5 |
|
6 |
## Llamacpp imatrix Quantizations of Qwen2.5-Coder-14B-Instruct
|
|
|
35 |
| [Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf) | Q5_K_M | 10.51GB | false | High quality, *recommended*. |
|
36 |
| [Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf) | Q5_K_S | 10.27GB | false | High quality, *recommended*. |
|
37 |
| [Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf) | Q4_K_L | 9.57GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
38 |
+
| [Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf) | Q4_K_M | 8.99GB | false | Good quality, default size for most use cases, *recommended*. |
|
39 |
| [Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf) | Q3_K_XL | 8.61GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
40 |
| [Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf) | Q4_K_S | 8.57GB | false | Slightly lower quality with more space savings, *recommended*. |
|
41 |
| [Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf) | IQ4_NL | 8.55GB | false | Similar to IQ4_XS, but slightly larger. |
|
|
|
119 |
|
120 |
## Credits
|
121 |
|
122 |
+
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
|
123 |
|
124 |
+
Thank you ZeroWw for the inspiration to experiment with embed/output.
|
125 |
|
126 |
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
|