MLDataScientist
/

Mistral-Large-Instruct-2407-GPTQ-3bit

Text Generation

Model card Files Files and versions Community

MLDataScientist commited on 14 days ago

Commit

afcb7cc

·

verified ·

1 Parent(s): b60184c

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -9,7 +9,7 @@ tags:
 This is a 3bit AutoRound GPTQ version of Mistral-Large-Instruct-2407.
 This conversion used model-*.safetensors.
-This quantized model needs at least ~50GB + context (~5GB) VRAM. I quantized it so that it could fit 64GB VRAM.
 Quantization script (it takes around 520 GB RAM and A40 GPU 48GB around 20 hours to convert):
 ```

 This is a 3bit AutoRound GPTQ version of Mistral-Large-Instruct-2407.
 This conversion used model-*.safetensors.
+This quantized model needs at least ~ 50GB + context (~5GB) VRAM. I quantized it so that it could fit 64GB VRAM.
 Quantization script (it takes around 520 GB RAM and A40 GPU 48GB around 20 hours to convert):
 ```