OPEA
/

DeepSeek-V3-int4-sym-awq-inc

4-bit precision

Model card Files Files and versions Community

cicdatopea commited on 8 days ago

Commit

e70fa45

·

verified ·

1 Parent(s): 28aa01a

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -274,7 +274,7 @@ https://github.com/intel/auto-round/blob/deepseekv3/modeling_deepseek.py
 ```bash
 git clone https://github.com/intel/auto-round.git && cd auto-round && git checkout deepseekv3
 ```
 ```bash
 python3 -m auto_round --model  "/models/DeepSeek-V3-bf16/"  --group_size 128 --format "auto_awq"  --iters 200 --devices 0,1,2,3,4 --nsamples 512 --batch_size 4 --seqlen 2048   --low_gpu_mem_usage    --output_dir "tmp_autoround"  --disable_eval e 2>&1 | tee -a seekv3.txt
 ```

 ```bash
 git clone https://github.com/intel/auto-round.git && cd auto-round && git checkout deepseekv3
 ```
+**torch<2.6**
 ```bash
 python3 -m auto_round --model  "/models/DeepSeek-V3-bf16/"  --group_size 128 --format "auto_awq"  --iters 200 --devices 0,1,2,3,4 --nsamples 512 --batch_size 4 --seqlen 2048   --low_gpu_mem_usage    --output_dir "tmp_autoround"  --disable_eval e 2>&1 | tee -a seekv3.txt
 ```