OPEA
/

QwQ-32B-int4-AutoRound-gptq-sym

4-bit precision

Model card Files Files and versions Community

cicdatopea commited on 18 days ago

Commit

93c20cc

·

verified ·

1 Parent(s): 1c4a4e7

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -192,7 +192,7 @@ auto-round --model "OPEA/QwQ-32B-int4-sym-gptq-inc" --eval --eval_bs 16  --tasks
 ### Generate the model
-Here is a sample command to generate the model. We found that the default parameters can cause issues with generation, though the lm-eval accuracy remains high. Please use the following command:
 ```bash
 auto-round \

 ### Generate the model
+Here is a sample command to generate the model.We found that this model is prone to overflow with FP16 int4 kernel. Please use the following command:
 ```bash
 auto-round \