text: Title: [Bug] Qwen2.5-VL-AWQ dose not support concurrent request
  Body: ### Checklist
  
  - [x] 1. I have searched related issues but cannot get the expected help.
  - [x] 2. The bug has not been fixed in the latest version.
  - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  - [x] 5. Please use English, otherwise it will be closed.
  
  ### Describe the bug
  
  Qwen2.5-VL-AWQ dose not support concurrent request
  
  ### Reproduction
  
  shell commond:
  ```
  modelPathOrName=$1
  tpSize=$2
  dpSize=$3
  portService=$4
  dockerName=$5
  CUDA_VISIBLE_DEVICES=$6
  image="lmsysorg/sglang:v0.4.0.post2-cu124"
  
  echo "gpus=$CUDA_VISIBLE_DEVICES"
  echo model:$modelPathOrName
  echo TP:$tpSize
  echo "DP:$dpSize"
  
  
  [ -z "$modelPathOrName" ] && modelPathOrName="Qwen2.5-VL-72B-Instruct-AWQ"
  [ -z "$CUDA_VISIBLE_DEVICES"  ] && CUDA_VISIBLE_DEVICES="2,3"
  [ ! -n "$tpSize" ] && tpSize=2
  [ ! -n "$dpSize" ] && dpSize=1
  [ ! -n "$portService" ] && portService=5090
  [ ! -n "$dockerName" ] && dockerName="qwen2.5-vl-awq"
  
  echo
  echo "finally..."
  echo model:$modelPathOrName
  echo TP:$tpSize
  echo "DP:$dpSize"
  
  dockerArgs=(
      --name "$dockerName"
      --gpus all
      --ipc host 
      --shm-size 64g 
      --network host
      --privileged
      -v /data/model:/home
      -w /home
      -d
      -e CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES
  )
  
  serverArgs=(
      --host 0.0.0.0
      --port $portService
      --model-path $modelPathOrName 
      --tp $tpSize 
      --dp $dpSize
      --load-balance-method round_robin
      --mem-fraction-static 0.8
      --enable-p2p-check
      --trust-remote-code
      --api-key sk-xxxxxxxxxxxx
      --chat-template qwen2-vl
  )
  
  docker run ${dockerArgs[@]} \
      $image \
      python3 -m sglang.launch_server \
      ${serverArgs[@]}
  ```
  
  ### Environment
  
  docker: `docker pull lmsysorg/sglang:v0.4.3.post2-cu124`
  model: https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct-AWQ