text: Title: [Bug] Why is the model not stopping and not producing any output? pls check screenshot.
  Body: ### Checklist
  
  - [x] 1. I have searched related issues but cannot get the expected help.
  - [ ] 2. The bug has not been fixed in the latest version.
  - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  - [x] 5. Please use English, otherwise it will be closed.
  
  ### Describe the bug
  
  here is my starting log : 
  
  ```
  python -m sglang.launch_server --model-path /home/drc-whlab/james/Qwen2___5-32B-Instruct-GPTQ-Int4 --host 0.0.0.0  --tp 2  --host 0.0.0.0 --port 7777 --max-running-requests 5 --dtype half --trust-remote-code --context-length 8192
  INFO 03-22 08:17:44 __init__.py:190] Automatically detected platform cuda.
  [2025-03-22 08:17:46] server_args=ServerArgs(model_path='/home/drc-whlab/james/Qwen2___5-32B-Instruct-GPTQ-Int4', tokenizer_path='/home/drc-whlab/james/Qwen2___5-32B-Instruct-GPTQ-Int4', tokenizer_mode='auto', skip_tokenizer_init=False, load_format='auto', trust_remote_code=True, dtype='half', kv_cache_dtype='auto', quantization=None, quantization_param_path=None, context_length=8192, device='cuda', served_model_name='/home/drc-whlab/james/Qwen2___5-32B-Instruct-GPTQ-Int4', chat_template=None, is_embedding=False, revision=None, host='0.0.0.0', port=7777, mem_fraction_static=0.87, max_running_requests=5, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='fcfs', schedule_conservativeness=1.0, cpu_offload_gb=0, page_size=1, tp_size=2, stream_interval=1, stream_output=False, random_seed=678263484, constrained_json_whitespace_pattern=None, watchdog_timeout=300, dist_timeout=None, download_dir=None, base_gpu_id=0, gpu_id_step=1, log_level='info', log_level_http=None, log_requests=False, log_requests_level=0, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_path='sglang_storage', enable_cache_report=False, reasoning_parser=None, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, lora_backend='triton', attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_algorithm=None, speculative_draft_model_path=None, speculative_num_steps=5, speculative_eagle_topk=4, speculative_num_draft_tokens=8, speculative_accept_threshold_single=1.0, speculative_accept_threshold_acc=1.0, speculative_token_map=None, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, enable_nccl_nvls=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, enable_custom_logit_processor=False, tool_call_parser=None, enable_hierarchical_cache=False, enable_flashinfer_mla=False, flashinfer_mla_disable_ragged=False, warmups=None, debug_tensor_dump_output_folder=None, debug_tensor_dump_input_file=None, debug_tensor_dump_inject=False)
  [2025-03-22 08:17:46] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
  INFO 03-22 08:17:49 __init__.py:190] Automatically detected platform cuda.
  INFO 03-22 08:17:49 __init__.py:190] Automatically detected platform cuda.
  INFO 03-22 08:17:49 __init__.py:190] Automatically detected platform cuda.
  [2025-03-22 08:17:51 TP1] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
  [2025-03-22 08:17:51 TP0] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
  [2025-03-22 08:17:51 TP1] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
  [2025-03-22 08:17:51 TP1] Init torch distributed begin.
  [2025-03-22 08:17:51 TP0] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
  [2025-03-22 08:17:51 TP0] Init torch distributed begin.
  [2025-03-22 08:17:52 TP0] sglang is using nccl==2.21.5
  [2025-03-22 08:17:52 TP1] sglang is using nccl==2.21.5
  [2025-03-22 08:17:52 TP0] Init torch distributed ends. mem usage=0.13 GB
  [2025-03-22 08:17:52 TP1] Init torch distributed ends. mem usage=0.13 GB
  [2025-03-22 08:17:52 TP0] Load weight begin. avail mem=15.31 GB
  [2025-03-22 08:17:52 TP1] Load weight begin. avail mem=15.29 GB
  [2025-03-22 08:17:52 TP0] Compute capability below sm80. Use float16 due to lack of bfloat16 support.
  [2025-03-22 08:17:52 TP1] Compute capability below sm80. Use float16 due to lack of bfloat16 support.
  [2025-03-22 08:17:52 TP0] The following error message 'operation scheduled before its operands' can be ignored.
  [2025-03-22 08:17:52 TP1] The following error message 'operation scheduled before its operands' can be ignored.
  Loading safetensors checkpoint shards:   0% Completed | 0/5 [00:00<?, ?it/s]
  Loading safetensors checkpoint shards:  20% Completed | 1/5 [00:00<00:02,  1.77it/s]
  Loading safetensors checkpoint shards:  40% Completed | 2/5 [00:01<00:01,  1.65it/s]
  Loading safetensors checkpoint shards:  60% Completed | 3/5 [00:02<00:01,  1.41it/s]
  Loading safetensors checkpoint shards:  80% Completed | 4/5 [00:02<00:00,  1.30it/s]
  Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:03<00:00,  1.25it/s]
  Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:03<00:00,  1.33it/s]
  
  [2025-03-22 08:17:57 TP0] Load weight end. type=Qwen2ForCausalLM, dtype=torch.float16, avail mem=6.12 GB, mem usage=9.19 GB.
  [2025-03-22 08:17:57 TP1] Load weight end. type=Qwen2ForCausalLM, dtype=torch.float16, avail mem=6.10 GB, mem usage=9.19 GB.
  [2025-03-22 08:17:57 TP0] KV Cache is allocated. #tokens: 33676, K size: 2.06 GB, V size: 2.06 GB
  [2025-03-22 08:17:57 TP1] KV Cache is allocated. #tokens: 33676, K size: 2.06 GB, V size: 2.06 GB
  [2025-03-22 08:17:57 TP0] Memory pool end. avail mem=1.87 GB
  [2025-03-22 08:17:57 TP1] Memory pool end. avail mem=1.85 GB
  [2025-03-22 08:17:57 TP0] Capture cuda graph begin. This can take up to several minutes. avail mem=1.25 GB
  [2025-03-22 08:17:57 TP1] Capture cuda graph begin. This can take up to several minutes. avail mem=1.24 GB
  100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:03<00:00,  1.54it/s]
  [2025-03-22 08:18:00 TP1] Registering 645 cuda graph addresses
  [2025-03-22 08:18:00 TP0] Registering 645 cuda graph addresses
  [2025-03-22 08:18:00 TP1] Capture cuda graph end. Time elapsed: 3.25 s. avail mem=0.82 GB. mem usage=0.42 GB.
  [2025-03-22 08:18:00 TP0] Capture cuda graph end. Time elapsed: 3.25 s. avail mem=0.84 GB. mem usage=0.42 GB.
  [2025-03-22 08:18:01 TP0] max_total_num_tokens=33676, chunked_prefill_size=2048, max_prefill_tokens=16384, max_running_requests=5, context_len=8192
  [2025-03-22 08:18:01 TP1] max_total_num_tokens=33676, chunked_prefill_size=2048, max_prefill_tokens=16384, max_running_requests=5, context_len=8192
  [2025-03-22 08:18:01] INFO:     Started server process [2560610]
  [2025-03-22 08:18:01] INFO:     Waiting for application startup.
  [2025-03-22 08:18:01] INFO:     Application startup complete.
  [2025-03-22 08:18:01] INFO:     Uvicorn running on http://0.0.0.0:7777 (Press CTRL+C to quit)
  [2025-03-22 08:18:02] INFO:     127.0.0.1:36748 - "GET /get_model_info HTTP/1.1" 200 OK
  [2025-03-22 08:18:02 TP0] Prefill batch. #new-seq: 1, #new-token: 6, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0,
  [2025-03-22 08:18:05] INFO:     127.0.0.1:36764 - "POST /generate HTTP/1.1" 200 OK
  [2025-03-22 08:18:05] The server is fired up and ready to roll!
  
  ```
  
  ### Reproduction
  
  ![Image](https://github.com/user-attachments/assets/51ae80ab-37da-4aca-82ed-685d083f7d3f)
  
  ### Environment
  
  
  ```
  pip list
  Package                           Version
  --------------------------------- -------------------
  aiohappyeyeballs                  2.6.1
  aiohttp                           3.11.14
  aiohttp-cors                      0.8.0
  aiosignal                         1.3.2
  airportsdata                      20250224
  annotated-types                   0.7.0
  anthropic                         0.49.0
  anyio                             4.9.0
  astor                             0.8.1
  asttokens                         3.0.0
  attrs                             25.3.0
  blake3                            1.0.4
  cachetools                        5.5.2
  certifi                           2025.1.31
  charset-normalizer                3.4.1
  click                             8.1.8
  cloudpickle                       3.1.1
  colorful                          0.5.6
  compressed-tensors                0.9.1
  cuda-bindings                     12.8.0
  cuda-python                       12.8.0
  datasets                          3.4.1
  decorator                         5.2.1
  decord                            0.6.0
  depyf                             0.18.0
  dill                              0.3.8
  diskcache                         5.6.3
  distlib                           0.3.9
  distro                            1.9.0
  einops                            0.8.1
  executing                         2.2.0
  fastapi                           0.115.11
  filelock                          3.18.0
  flashinfer-python                 0.2.3+cu124torch2.5
  frozenlist                        1.5.0
  fsspec                            2024.12.0
  gguf                              0.10.0
  google-api-core                   2.24.2
  google-auth                       2.38.0
  googleapis-common-protos          1.69.2
  grpcio                            1.71.0
  h11                               0.14.0
  hf_transfer                       0.1.9
  httpcore                          1.0.7
  httptools                         0.6.4
  httpx                             0.28.1
  huggingface-hub                   0.29.3
  idna                              3.10
  importlib_metadata                8.6.1
  interegular                       0.3.3
  ipython                           9.0.2
  ipython_pygments_lexers           1.1.1
  jedi                              0.19.2
  Jinja2                            3.1.6
  jiter                             0.9.0
  jsonschema                        4.23.0
  jsonschema-specifications         2024.10.1
  lark                              1.2.2
  litellm                           1.63.12
  llguidance                        0.7.5
  lm-format-enforcer                0.10.11
  MarkupSafe                        3.0.2
  matplotlib-inline                 0.1.7
  mistral_common                    1.5.4
  modelscope                        1.24.0
  mpmath                            1.3.0
  msgpack                           1.1.0
  msgspec                           0.19.0
  multidict                         6.2.0
  multiprocess                      0.70.16
  nest-asyncio                      1.6.0
  networkx                          3.4.2
  ninja                             1.11.1.3
  numpy                             1.26.4
  nvidia-cublas-cu12                12.4.5.8
  nvidia-cuda-cupti-cu12            12.4.127
  nvidia-cuda-nvrtc-cu12            12.4.127
  nvidia-cuda-runtime-cu12          12.4.127
  nvidia-cudnn-cu12                 9.1.0.70
  nvidia-cufft-cu12                 11.2.1.3
  nvidia-curand-cu12                10.3.5.147
  nvidia-cusolver-cu12              11.6.1.9
  nvidia-cusparse-cu12              12.3.1.170
  nvidia-ml-py                      12.570.86
  nvidia-nccl-cu12                  2.21.5
  nvidia-nvjitlink-cu12             12.4.127
  nvidia-nvtx-cu12                  12.4.127
  openai                            1.67.0
  opencensus                        0.11.4
  opencensus-context                0.1.3
  opencv-python-headless            4.11.0.86
  orjson                            3.10.15
  outlines                          0.1.11
  outlines_core                     0.1.26
  packaging                         24.2
  pandas                            2.2.3
  parso                             0.8.4
  partial-json-parser               0.2.1.1.post5
  pexpect                           4.9.0
  pillow                            11.1.0
  pip                               25.0.1
  platformdirs                      4.3.7
  prometheus_client                 0.21.1
  prometheus-fastapi-instrumentator 7.1.0
  prompt_toolkit                    3.0.50
  propcache                         0.3.0
  proto-plus                        1.26.1
  protobuf                          6.30.1
  psutil                            7.0.0
  ptyprocess                        0.7.0
  pure_eval                         0.2.3
  py-cpuinfo                        9.0.0
  py-spy                            0.4.0
  pyarrow                           19.0.1
  pyasn1                            0.6.1
  pyasn1_modules                    0.4.1
  pycountry                         24.6.1
  pydantic                          2.10.6
  pydantic_core                     2.27.2
  Pygments                          2.19.1
  python-dateutil                   2.9.0.post0
  python-dotenv                     1.0.1
  python-multipart                  0.0.20
  pytz                              2025.1
  PyYAML                            6.0.2
  pyzmq                             26.3.0
  ray                               2.44.0
  referencing                       0.36.2
  regex                             2024.11.6
  requests                          2.32.3
  rpds-py                           0.23.1
  rsa                               4.9
  safetensors                       0.5.3
  sentencepiece                     0.2.0
  setproctitle                      1.3.5
  setuptools                        75.8.0
  sgl-kernel                        0.0.5
  sglang                            0.4.4
  six                               1.17.0
  smart-open                        7.1.0
  sniffio                           1.3.1
  stack-data                        0.6.3
  starlette                         0.46.1
  sympy                             1.13.1
  tiktoken                          0.9.0
  tokenizers                        0.21.1
  torch                             2.5.1
  torchao                           0.9.0
  torchaudio                        2.5.1
  torchvision                       0.20.1
  tqdm                              4.67.1
  traitlets                         5.14.3
  transformers                      4.48.3
  triton                            3.1.0
  typing_extensions                 4.12.2
  tzdata                            2025.1
  urllib3                           2.3.0
  uv                                0.6.9
  uvicorn                           0.34.0
  uvloop                            0.21.0
  virtualenv                        20.29.3
  vllm                              0.7.2
  watchfiles                        1.0.4
  wcwidth                           0.2.13
  websockets                        15.0.1
  wheel                             0.45.1
  wrapt                             1.17.2
  xformers                          0.0.28.post3
  xgrammar                          0.1.15
  xxhash                            3.5.0
  yarl                              1.18.3
  zipp                              3.21.0
  
  ```