gemma-3-4b-it-abliterated.q8q4.gguf is very much appreciated.
#1
by
twoxfh
- opened
My old igpu laptop appreciates gemma-3-4b-it-abliterated.q8q4.gguf; are you sharing the code of how its made? Curious to see it applied to other models.
Sure:
- convert the model to GGUF F16.
- quantize the model to q4 but use q8 for the output and embed tensors. (quantize utility has the option for this).
My old igpu laptop appreciates gemma-3-4b-it-abliterated.q8q4.gguf; are you sharing the code of how its made? Curious to see it applied to other models.
check my models, I converted most of them already. https://huggingface.co/ZeroWw
ZeroWw
changed discussion status to
closed