Error generating gguf quantization
#1
by
Doctor-Chad-PhD
- opened
Hi, thank you for this interesting model!
I'm trying to generate quants for this model, but I'm encountering some issues. The 'quantize' script: quantize_models.sh appears to rely on a pre-computed BF16 quantization, which I can't generate directly. I'm getting two errors using convert_hf_to_gguf.py (from llama.cpp):
- A 'FileNotFoundError: File not found: tokenizer.model' error, suggesting that the model I'm trying to quantize is looking for a different file.
- A 'Exception: EOF while parsing a value at line 1 column 0' error related to the vocab.
Can you help me quantize this model myself without relying on pre-computed BF16 quantization? Thanks!
Sorry, this was my own error. My download was corrupted.
Doctor-Chad-PhD
changed discussion status to
closed
it's a valid concern. I've fixed the script to do this automatically as it was just a comment at the top prior. If you git pull it should be there
Thank you!