Intel
/

Qwen2.5-VL-7B-Instruct-int4-mixed-AutoRound

4-bit precision

Model card Files Files and versions

wenhuach commited on 7 days ago

Commit

af9a2ee

·

verified ·

1 Parent(s): 4c089e3

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -8,7 +8,7 @@ this pr https://github.com/vllm-project/vllm/pull/21802 is required.
  ~~~bash
- vllm serve Intel/Qwen2.5-VL-7B-Instruct-int4-mixed-AutoRound  --dtype bfloat16   --max-model-len 10000
 ~~~
 ~~~bash

  ~~~bash
+ vllm serve Intel/Qwen2.5-VL-7B-Instruct-int4-mixed-AutoRound  --dtype bfloat16 --port 8001  --max-model-len 10000
 ~~~
 ~~~bash