Spaces:
Running
on
A10G
Running
on
A10G
Please, update converting script. Llama.cpp added support for Nemotron and Minitron architectures.
#111
by
NikolayKozloff
- opened
I tried to make a gguf for nvidia/Nemotron-4-Minitron-8B-Base using your HF space but got this error: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: Nemotron-4-Minitron-8B-Base\nERROR:hf-to-gguf:Model NemotronForCausalLM is not supported\n'
The space is restarted and it should be running the latest version now
ngxson
changed discussion status to
closed
The space is restarted and it should be running the latest version now
It still doesn't work, i got the same error message again.
reach-vb
changed discussion status to
open
Hi @NikolayKozloff - there are known issues with the 8B: https://github.com/ggerganov/llama.cpp/pull/8922#issuecomment-2294891881, will require a fix in transformers, should be soon.