will there be a GGUF?

#1
by UniversalLove333 - opened

Thanks for your interest in our work! We'll provide a GGUF version as soon as possible.

Hi! Thanks for your interest in the model. 🙌

We’ve just uploaded the GGUF version as well as an AWQ quantized version for convenience and broader compatibility.

That said, for best performance and fidelity, I highly recommend using the original full model (the base version), as quantization may lead to slight degradation in accuracy, especially on complex tasks.

Let me know if you run into any issues or need help getting started!

Sign up or log in to comment