Quant with Llamacpp ?

#4
by DavidAU - opened

Your model is listed as source for "abliterated" version:

https://huggingface.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated

This version will not quant in Llamacpp ; error "convert_to_gguf..."

line 8019, in generate_extra_tensors
raise ValueError("No MXFP4 tensors found in the model. Please make sure you are using MXFP4 model.")
ValueError: No MXFP4 tensors found in the model. Please make sure you are using MXFP4 model.

(also tried older Llamacpp versions - no joy)

Will your version - "normal" - here work?
thanks ;

Sorry for the late reply, you need to recompile llama.cpp from scratch, it should work yes

Sign up or log in to comment