Internal Server Error 500 in VLLM while using some images.
#12
by
as-sriram
- opened
Hosting the model via command:
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m vllm.entrypoints.openai.api_server --model /models/Qwen/Qwen2.5-VL-32B-Instruct --dtype auto --tensor-parallel-size 4 --served-model-name qwen_2.5_32b_vl --max_model_len 128000 --tokenizer_mode auto --limit-mm-per-prompt image\=10 --disable-log-requests
Here are the sample list of images which are leading to the error. Can someone please help me with the issue. Thanks.
as-sriram
changed discussion title from
Internal Server error 500 in VLLM while using some images.
to Internal Server Error 500 in VLLM while using some images.
got the same problem there