Hugging Face's logo

DavidAU
/
Qwen2.5-The-Wisemen-QwQ-Deep-Tiny-Sherlock-32B-GGUF

GGUF
QwQ-32B
reasoning
thinking
r1
cot
deepseek
Qwen2.5
Hermes
DeepHermes
DeepSeek
DeepSeek-R1-Distill
128k context
Merge
Uncensored
creative
general usage
problem solving
brainstorming
solve riddles
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
story
writing
fiction
roleplaying
swearing
horror
Qwen 2.5
mergekit
conversational
Model card Files Community
1
DavidAU mradermacher commited on
Commit
f8a908a
·
verified ·
0 Parent(s):

Duplicate from mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF

Browse files

Co-authored-by: team mradermacher <[email protected]>

Files changed (13) hide show
  1. .gitattributes +45 -0
  2. Qwen2.5-QwQ-32B-Four-Banger.IQ4_XS.gguf +3 -0
  3. Qwen2.5-QwQ-32B-Four-Banger.Q2_K.gguf +3 -0
  4. Qwen2.5-QwQ-32B-Four-Banger.Q3_K_L.gguf +3 -0
  5. Qwen2.5-QwQ-32B-Four-Banger.Q3_K_M.gguf +3 -0
  6. Qwen2.5-QwQ-32B-Four-Banger.Q3_K_S.gguf +3 -0
  7. Qwen2.5-QwQ-32B-Four-Banger.Q4_K_M.gguf +3 -0
  8. Qwen2.5-QwQ-32B-Four-Banger.Q4_K_S.gguf +3 -0
  9. Qwen2.5-QwQ-32B-Four-Banger.Q5_K_M.gguf +3 -0
  10. Qwen2.5-QwQ-32B-Four-Banger.Q5_K_S.gguf +3 -0
  11. Qwen2.5-QwQ-32B-Four-Banger.Q6_K.gguf +3 -0
  12. Qwen2.5-QwQ-32B-Four-Banger.Q8_0.gguf +3 -0
  13. README.md +74 -0
.gitattributes ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Qwen2.5-QwQ-32B-Four-Banger.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Qwen2.5-QwQ-32B-Four-Banger.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Qwen2.5-QwQ-32B-Four-Banger.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Qwen2.5-QwQ-32B-Four-Banger.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Qwen2.5-QwQ-32B-Four-Banger.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Qwen2.5-QwQ-32B-Four-Banger.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Qwen2.5-QwQ-32B-Four-Banger.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Qwen2.5-QwQ-32B-Four-Banger.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Qwen2.5-QwQ-32B-Four-Banger.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Qwen2.5-QwQ-32B-Four-Banger.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-QwQ-32B-Four-Banger.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2100c20cda8fa58c650dc4ba72cb4d1cb76c20b254e5a2cad5a0d2aa64fa4cf
3
+ size 17867359872
Qwen2.5-QwQ-32B-Four-Banger.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:add2f141ce19c467e6b496c14d28543fff523ac52e5383316e22268510c3e261
3
+ size 12310768448
Qwen2.5-QwQ-32B-Four-Banger.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8565523440ffe0770a520b758fdf49f2e8bf24a8a0afb33d53e7f75f622aa544
3
+ size 17244543200
Qwen2.5-QwQ-32B-Four-Banger.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd95004dc70dfe6e321d23e93118ceeef6df7ca145f44d45d379d13f4183e771
3
+ size 15932512480
Qwen2.5-QwQ-32B-Four-Banger.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4ee83febc553921b547afbc336bc6b6ede47985a109b455ade1ba00f98a5a46
3
+ size 14389795040
Qwen2.5-QwQ-32B-Four-Banger.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5ee74b29fdedb1598735509b5c795e77ac9de676c0b571452b9253867e25f7b8
3
+ size 19848531872
Qwen2.5-QwQ-32B-Four-Banger.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd22c7d520e6697d8cbd9c52fbd35d269d51599f65121e240bcfa5c035b8772e
3
+ size 18781605792
Qwen2.5-QwQ-32B-Four-Banger.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60f8c4048371c79a6fba828e7cfc3c4dd78c4adc838e509ee433afde83be5002
3
+ size 23259100192
Qwen2.5-QwQ-32B-Four-Banger.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d38a2acf148eb723a021a9839b8f1579b65ac72df1aa419d9c819dfa3455acb
3
+ size 22635197472
Qwen2.5-QwQ-32B-Four-Banger.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcd9741646c3270d7afe4e7f67fa1477d4446f9d7f3fc47d33731f8d2cb0660a
3
+ size 26882829056
Qwen2.5-QwQ-32B-Four-Banger.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24fb8a1c3ba5b221e6b42b1eccf65d897ad9014d93d0faf12bc708e8684ea486
3
+ size 34816579648
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/Qwen2.5-QwQ-32B-Four-Banger
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ quantized_by: mradermacher
7
+ tags:
8
+ - QwQ-32B
9
+ - reasoning
10
+ - thinking
11
+ - r1
12
+ - cot
13
+ - deepseek
14
+ - Qwen2.5
15
+ - Hermes
16
+ - DeepHermes
17
+ - DeepSeek
18
+ - DeepSeek-R1-Distill
19
+ - 128k context
20
+ - merge
21
+ ---
22
+ ## About
23
+
24
+ <!-- ### quantize_version: 2 -->
25
+ <!-- ### output_tensor_quantised: 1 -->
26
+ <!-- ### convert_type: hf -->
27
+ <!-- ### vocab_type: -->
28
+ <!-- ### tags: -->
29
+ static quants of https://huggingface.co/DavidAU/Qwen2.5-QwQ-32B-Four-Banger
30
+
31
+ <!-- provided-files -->
32
+ weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
33
+ ## Usage
34
+
35
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
36
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
37
+ more details, including on how to concatenate multi-part files.
38
+
39
+ ## Provided Quants
40
+
41
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
42
+
43
+ | Link | Type | Size/GB | Notes |
44
+ |:-----|:-----|--------:|:------|
45
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q2_K.gguf) | Q2_K | 12.4 | |
46
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q3_K_S.gguf) | Q3_K_S | 14.5 | |
47
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q3_K_M.gguf) | Q3_K_M | 16.0 | lower quality |
48
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q3_K_L.gguf) | Q3_K_L | 17.3 | |
49
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q4_K_S.gguf) | Q4_K_S | 18.9 | fast, recommended |
50
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q4_K_M.gguf) | Q4_K_M | 19.9 | fast, recommended |
51
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q5_K_S.gguf) | Q5_K_S | 22.7 | |
52
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q6_K.gguf) | Q6_K | 27.0 | very good quality |
53
+ | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-QwQ-32B-Four-Banger-GGUF/resolve/main/Qwen2.5-QwQ-32B-Four-Banger.Q8_0.gguf) | Q8_0 | 34.9 | fast, best quality |
54
+
55
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
56
+ types (lower is better):
57
+
58
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
59
+
60
+ And here are Artefact2's thoughts on the matter:
61
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
62
+
63
+ ## FAQ / Model Request
64
+
65
+ See https://huggingface.co/mradermacher/model_requests for some answers to
66
+ questions you might have and/or if you want some other model quantized.
67
+
68
+ ## Thanks
69
+
70
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
71
+ me use its servers and providing upgrades to my workstation to enable
72
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
73
+
74
+ <!-- end -->