Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,12 @@ pipeline_tag: text-generation
|
|
9 |
## Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ
|
10 |
This is a version of the Mixtral-8x7B-Instruct-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) quantized to 4-bit via Half-Quadratic Quantization (HQQ).
|
11 |
### Basic Usage
|
12 |
-
To run the model, install the HQQ library
|
|
|
|
|
|
|
|
|
|
|
13 |
``` Python
|
14 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ'
|
15 |
|
|
|
9 |
## Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ
|
10 |
This is a version of the Mixtral-8x7B-Instruct-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) quantized to 4-bit via Half-Quadratic Quantization (HQQ).
|
11 |
### Basic Usage
|
12 |
+
To run the model, install the HQQ library:
|
13 |
+
```
|
14 |
+
pip install hqq==0.1.8
|
15 |
+
pip install transformers==4.46.0
|
16 |
+
```
|
17 |
+
and use it as follows:
|
18 |
``` Python
|
19 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ'
|
20 |
|