Where are the QAT releases for Gemma 3?
#28
by
Downtown-Case
- opened
I see the flax format on Kaggle. And the gemma.cpp FP8 version.
Is that the only form the FP8/int4 QAT weights are released? Is there an official GGUF, or are the any plans for huggingface-format QAT releases for Gemma 3?
These are the official GGUF int4 QAT models:
https://huggingface.co/collections/google/gemma-3-qat-67ee61ccacbf2be4195c265b
While the models are hosted on Hugging Face, it appears they cannot run via their Transformers package. You'll have to use other packages like Ollama.