openerotica
/

c4ai-command-r-plus-GPTQ-ERQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Where can we download `quant.py`?

#1

by shuyuej - opened Jul 21, 2024

shuyuej

Jul 21, 2024

Where can we download quant.py?

basiliskinstitute

openerotica org Jul 25, 2024

Try one of these:

https://gist.github.com/CheshireAI/608c64061b73098fb7a1ea8392ce2b81
https://gist.github.com/CheshireAI/480222ef2065cc2ec75f9727addc2222

It's basically TheBloke's script with added support for a hardcoded custom dataset.
Original: https://gist.github.com/TheBloke/b47c50a70dd4fe653f64a12928286682

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment