7b?
#2
by
LaferriereJC
- opened
so far this is the only version I can run of gemma
Hi LaferriereJC, thanks for reaching out about this. We're a bit suspect of the 7B gguf file that google has uploaded (and we would quantize).
As a result, we're holding off on an upload until we get more information on that. See https://huggingface.co/google/gemma-7b-it/discussions/38 for a detailed explanation of us reaching out to google about the issues we're seeing.
@mattjcly llamacpp recently updated to be able to properly quantize the models. Would really appreciate and update 😁
Hi, when will we have 7b-it gguf?