Update README.md
Browse files
README.md
CHANGED
@@ -22,9 +22,11 @@ This is version 1. It has been fine-tuned using a subset of the data from Pygmal
|
|
22 |
This models has the XOR files pre-applied out of the box.
|
23 |
Converted from the XORs weights from PygmalionAI's release https://huggingface.co/PygmalionAI/pygmalion-7b
|
24 |
|
25 |
-
|
|
|
|
|
26 |
```
|
27 |
-
python llama.py
|
28 |
```
|
29 |
|
30 |
## Prompting
|
|
|
22 |
This models has the XOR files pre-applied out of the box.
|
23 |
Converted from the XORs weights from PygmalionAI's release https://huggingface.co/PygmalionAI/pygmalion-7b
|
24 |
|
25 |
+
Quantization was done using https://github.com/0cc4m/GPTQ-for-LLaMa for use in KoboldAI
|
26 |
+
|
27 |
+
Via the following command:
|
28 |
```
|
29 |
+
python llama.py ./TehVenom_Pygmalion-7b-Merged-Safetensors c4 --wbits 4 --true-sequential --groupsize 32 --save_safetensors Pygmalion-7B-GPTQ-4bit-32g.no-act-order.safetensors
|
30 |
```
|
31 |
|
32 |
## Prompting
|