bartowski commited on
Commit
7f20e8a
·
verified ·
1 Parent(s): d8cd165

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -1,7 +1,6 @@
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
4
- base_model: Qwen/Qwen2.5-Coder-14B-Instruct
5
  ---
6
 
7
  ## Llamacpp imatrix Quantizations of Qwen2.5-Coder-14B-Instruct
@@ -36,7 +35,7 @@ Run them in [LM Studio](https://lmstudio.ai/)
36
  | [Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf) | Q5_K_M | 10.51GB | false | High quality, *recommended*. |
37
  | [Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf) | Q5_K_S | 10.27GB | false | High quality, *recommended*. |
38
  | [Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf) | Q4_K_L | 9.57GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
39
- | [Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf) | Q4_K_M | 8.99GB | false | Good quality, default size for must use cases, *recommended*. |
40
  | [Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf) | Q3_K_XL | 8.61GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
41
  | [Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf) | Q4_K_S | 8.57GB | false | Slightly lower quality with more space savings, *recommended*. |
42
  | [Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf) | IQ4_NL | 8.55GB | false | Similar to IQ4_XS, but slightly larger. |
@@ -120,8 +119,8 @@ The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have
120
 
121
  ## Credits
122
 
123
- Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
124
 
125
- Thank you ZeroWw for the inspiration to experiment with embed/output
126
 
127
  Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
 
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
 
4
  ---
5
 
6
  ## Llamacpp imatrix Quantizations of Qwen2.5-Coder-14B-Instruct
 
35
  | [Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_M.gguf) | Q5_K_M | 10.51GB | false | High quality, *recommended*. |
36
  | [Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q5_K_S.gguf) | Q5_K_S | 10.27GB | false | High quality, *recommended*. |
37
  | [Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_L.gguf) | Q4_K_L | 9.57GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
38
+ | [Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf) | Q4_K_M | 8.99GB | false | Good quality, default size for most use cases, *recommended*. |
39
  | [Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q3_K_XL.gguf) | Q3_K_XL | 8.61GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
40
  | [Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-Q4_K_S.gguf) | Q4_K_S | 8.57GB | false | Slightly lower quality with more space savings, *recommended*. |
41
  | [Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/blob/main/Qwen2.5-Coder-14B-Instruct-IQ4_NL.gguf) | IQ4_NL | 8.55GB | false | Similar to IQ4_XS, but slightly larger. |
 
119
 
120
  ## Credits
121
 
122
+ Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
123
 
124
+ Thank you ZeroWw for the inspiration to experiment with embed/output.
125
 
126
  Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski