JayhC
/

Yi-34B-200K-RPMerge-6bpw-h6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

JayhC commited on Mar 26, 2024

Commit

06b68d6

·

verified ·

1 Parent(s): f96abff

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -17,7 +17,7 @@ tags:
 .
-6bpw/h6 exl2 quantization of [brucethemoose/Yi-34B-200K-RPMerge](https://huggingface.co/brucethemoose/Yi-34B-200K-RPMerge) using default exllamav2 calibration dataset, to fully use my 31gb VRAM (-1 cuz windows..).
 ---

 .
+6bpw/h6 exl2 quantization of [brucethemoose/Yi-34B-200K-RPMerge](https://huggingface.co/brucethemoose/Yi-34B-200K-RPMerge) using default exllamav2 calibration dataset, to fully use my 31gb VRAM (-1 cuz windows..) with 16k-32k context.
 ---