Hugging Face's logo

Kunger
/
Sakura-13B-Qwen2beta-v0.9-4bit-GS64-AWQ

Text Generation
Transformers
Safetensors
qwen2
conversational
text-generation-inference
Inference Endpoints
4-bit precision
awq
Model card Files Community
Kunger commited on
Commit
ce71c3a
·
verified ·
1 Parent(s): 0b47031

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -8,4 +8,4 @@ license: cc-by-nc-sa-4.0
8
 
9
  GroupSize=64
10
 
11
- vLLM双卡推理不兼容AWQ，查ISSUE说好像量化时GroupSize设置为64可以解决。
 
8
 
9
  GroupSize=64
10
 
11
+ 适用于Kaggle双卡推理。