Hugging Face's logo

mradermacher
/
Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF

Transformers
GGUF
creative
creative writing
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prose
vivid writing
fiction
roleplaying
bfloat16
swearing
rp
qwen3
horror
finetune
Merge
imatrix
conversational
Model card Files
xet
Community
mradermacher commited on
Commit
3059d03
·
verified ·
1 Parent(s): a71f7fe

auto-patch README.md

Browse files
Files changed (1) hide show
  1. README.md +100 -0
README.md CHANGED
@@ -1,6 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!-- ### quantize_version: 2 -->
2
  <!-- ### output_tensor_quantised: 1 -->
3
  <!-- ### convert_type: hf -->
4
  <!-- ### vocab_type: -->
5
  <!-- ### tags: nicoboss -->
6
  weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000
3
+ language:
4
+ - en
5
+ - fr
6
+ - zh
7
+ - de
8
+ library_name: transformers
9
+ license: apache-2.0
10
+ quantized_by: mradermacher
11
+ tags:
12
+ - creative
13
+ - creative writing
14
+ - fiction writing
15
+ - plot generation
16
+ - sub-plot generation
17
+ - fiction writing
18
+ - story generation
19
+ - scene continue
20
+ - storytelling
21
+ - fiction story
22
+ - science fiction
23
+ - romance
24
+ - all genres
25
+ - story
26
+ - writing
27
+ - vivid prose
28
+ - vivid writing
29
+ - fiction
30
+ - roleplaying
31
+ - bfloat16
32
+ - swearing
33
+ - rp
34
+ - qwen3
35
+ - horror
36
+ - finetune
37
+ - merge
38
+ ---
39
+ ## About
40
+
41
  <!-- ### quantize_version: 2 -->
42
  <!-- ### output_tensor_quantised: 1 -->
43
  <!-- ### convert_type: hf -->
44
  <!-- ### vocab_type: -->
45
  <!-- ### tags: nicoboss -->
46
  weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000
47
+
48
+ <!-- provided-files -->
49
+ static quants are available at https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-GGUF
50
+ ## Usage
51
+
52
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
53
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
54
+ more details, including on how to concatenate multi-part files.
55
+
56
+ ## Provided Quants
57
+
58
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
59
+
60
+ | Link | Type | Size/GB | Notes |
61
+ |:-----|:-----|--------:|:------|
62
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf) | i1-IQ1_S | 1.2 | for the desperate |
63
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate |
64
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.3 | |
65
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | |
67
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf) | i1-IQ2_M | 1.6 | |
68
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality |
69
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better |
70
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.8 | lower quality |
71
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.9 | |
72
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.0 | IQ3_XS probably better |
73
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf) | i1-IQ3_S | 2.0 | beats Q3_K* |
74
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf) | i1-IQ3_M | 2.1 | |
75
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.2 | IQ3_S probably better |
76
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.3 | IQ3_M probably better |
77
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.4 | |
78
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf) | i1-Q4_0 | 2.5 | fast, low quality |
79
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.5 | prefer IQ4_XS |
80
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.5 | optimal size/speed/quality |
81
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.6 | fast, recommended |
82
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf) | i1-Q4_1 | 2.7 | |
83
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.9 | |
84
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.0 | |
85
+ | [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf) | i1-Q6_K | 3.4 | practically like static Q6_K |
86
+
87
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
88
+ types (lower is better):
89
+
90
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
91
+
92
+ And here are Artefact2's thoughts on the matter:
93
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
94
+
95
+ ## FAQ / Model Request
96
+
97
+ See https://huggingface.co/mradermacher/model_requests for some answers to
98
+ questions you might have and/or if you want some other model quantized.
99
+
100
+ ## Thanks
101
+
102
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
103
+ me use its servers and providing upgrades to my workstation to enable
104
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
105
+
106
+ <!-- end -->