Update README.md
Browse files
README.md
CHANGED
@@ -15,7 +15,12 @@ This model should perform a lot better compared to the all 2-bit model for a sli
|
|
15 |
This idea was suggest by Artem Eliseev (@lavawolfiee) and Denis Mazur (@dvmazur) [in this Github discussion](https://github.com/mobiusml/hqq/issues/2).
|
16 |
|
17 |
### Basic Usage
|
18 |
-
To run the model, install the HQQ library
|
|
|
|
|
|
|
|
|
|
|
19 |
``` Python
|
20 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ'
|
21 |
#Load the model
|
|
|
15 |
This idea was suggest by Artem Eliseev (@lavawolfiee) and Denis Mazur (@dvmazur) [in this Github discussion](https://github.com/mobiusml/hqq/issues/2).
|
16 |
|
17 |
### Basic Usage
|
18 |
+
To run the model, install the HQQ library:
|
19 |
+
```
|
20 |
+
pip install hqq==0.1.8
|
21 |
+
pip install transformers==4.46.0
|
22 |
+
```
|
23 |
+
and use it as follows:
|
24 |
``` Python
|
25 |
model_id = 'mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ'
|
26 |
#Load the model
|