Quant with Llamacpp ?
#4
by
DavidAU
- opened
Your model is listed as source for "abliterated" version:
https://huggingface.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated
This version will not quant in Llamacpp ; error "convert_to_gguf..."
line 8019, in generate_extra_tensors
raise ValueError("No MXFP4 tensors found in the model. Please make sure you are using MXFP4 model.")
ValueError: No MXFP4 tensors found in the model. Please make sure you are using MXFP4 model.
(also tried older Llamacpp versions - no joy)
Will your version - "normal" - here work?
thanks ;
Sorry for the late reply, you need to recompile llama.cpp from scratch, it should work yes