quant this pls
#1
by
Utochi
- opened
I tried to quantize to MXFP4 and got errors
Going down to Q4_1 worked but the output was very very bad
making the imatrix resulted in a bunch of NaN, so no smaller sizes coming yet.
Seems like it may have been bad luck, or just a problem with Q4_1.
Q4_K_M and MXFP4 both look way better, I've uploaded them.
Still no imatrix though.
2 Imats here ; more on the way:
https://huggingface.co/DavidAU/OpenAi-GPT-oss-20b-abliterated-uncensored-NEO-Imatrix-gguf