text: Title: [Bug]  Please help! I encountered an error when using the command to deploy the model, but there was no error message.
  Body: ### Checklist
  
  - [ ] 1. I have searched related issues but cannot get the expected help.
  - [ ] 2. The bug has not been fixed in the latest version.
  - [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  - [ ] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  - [ ] 5. Please use English, otherwise it will be closed.
  
  ### Describe the bug
  
  
  I got error
  
  `/home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  INFO 02-22 01:37:05 __init__.py:190] Automatically detected platform cuda.
  [2025-02-22 01:37:14] server_args=ServerArgs(model_path='sophosympatheia/Midnight-Miqu-70B-v1.0', tokenizer_path='sophosympatheia/Midnight-Miqu-70B-v1.0', tokenizer_mode='auto', load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=None, device='cuda', served_model_name='sophosympatheia/Midnight-Miqu-70B-v1.0', chat_template=None, is_embedding=False, revision=None, skip_tokenizer_init=False, host='127.0.0.1', port=30000, mem_fraction_static=0.9, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=8192, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=8, stream_interval=1, stream_output=False, random_seed=288994143, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http=None, log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, lora_backend='triton', attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, enable_nccl_nvls=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=160, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=False, return_hidden_states=False, enable_custom_logit_processor=False, tool_call_parser=None, enable_hierarchical_cache=False, enable_flashinfer_mla=False)
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  /home/ubuntu/sglang/lib/python3.10/site-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
    warnings.warn(
  INFO 02-22 01:37:18 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:18 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  INFO 02-22 01:37:19 __init__.py:190] Automatically detected platform cuda.
  [2025-02-22 01:37:30 TP2] Init torch distributed begin.
  [2025-02-22 01:37:32 TP7] Init torch distributed begin.
  [2025-02-22 01:37:32 TP3] Init torch distributed begin.
  [2025-02-22 01:37:32 TP0] Init torch distributed begin.
  [2025-02-22 01:37:32 TP4] Init torch distributed begin.
  [2025-02-22 01:37:32 TP5] Init torch distributed begin.
  [2025-02-22 01:37:32 TP6] Init torch distributed begin.
  [2025-02-22 01:37:32 TP1] Init torch distributed begin.
  [2025-02-22 01:37:33 TP5] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP1] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP0] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP2] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP3] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP4] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP7] sglang is using nccl==2.21.5
  [2025-02-22 01:37:33 TP6] sglang is using nccl==2.21.5
  Fatal Python error: Segmentation fault
  
  Current thread 0x000072862798f1c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  [2025-02-22 01:37:39] Rank 0 scheduler is dead. Please check if there are relevant logs.
  Fatal Python error: Segmentation fault
  
  Current thread 0x00007c21b539c1c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  Fatal Python error: Segmentation fault
  
  Current thread 0x000078dfe49a51c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  Fatal Python error: Segmentation fault
  
  Current thread 0x0000763bcdcde1c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  Fatal Python error: Segmentation fault
  
  Current thread 0x0000726fb547b1c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  [2025-02-22 01:37:39] Exit code: -11
  Fatal Python error: Segmentation fault
  
  Current thread 0x000071453395c1c0 (most recent call first):
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl_wrapper.py", line 351 in ncclCommInitRank
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/device_communicators/pynccl.py", line 108 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 233 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 890 in init_model_parallel_group
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py", line 1055 in initialize_model_parallel
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 262 in init_torch_distributed
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 187 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 63 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 240 in __init__
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1816 in run_scheduler_process
    File "/usr/lib/python3.10/multiprocessing/process.py", line 108 in run
    File "/usr/lib/python3.10/multiprocessing/process.py", line 314 in _bootstrap
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 129 in _main
    File "/usr/lib/python3.10/multiprocessing/spawn.py", line 116 in spawn_main
    File "<string>", line 1 in <module>
  
  Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, uvloop.loop, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, psutil._psutil_posix, setproctitle, zmq.backend.cython._zmq, yaml._yaml, markupsafe._speedups, PIL._imaging, PIL._imagingft, msgspec._core, msgpack._cmsgpack, google._upb._message, ray._raylet, sentencepiece._sentencepiece, regex._regex (total: 50)
  Traceback (most recent call last):
    File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
      exec(code, run_globals)
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/launch_server.py", line 14, in <module>
      launch_server(server_args)
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/entrypoints/http_server.py", line 491, in launch_server
      tokenizer_manager, scheduler_info = _launch_subprocesses(server_args=server_args)
    File "/home/ubuntu/sglang/lib/python3.10/site-packages/sglang/srt/entrypoints/engine.py", line 449, in _launch_subprocesses
      data = scheduler_pipe_readers[i].recv()
    File "/usr/lib/python3.10/multiprocessing/connection.py", line 250, in recv
      buf = self._recv_bytes()
    File "/usr/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes
      buf = self._recv(4)
    File "/usr/lib/python3.10/multiprocessing/connection.py", line 383, in _recv
      raise EOFError
  EOFError`
  
  ### Reproduction
  
  I use the commend ` python -m sglang.launch_server --model-path sophosympatheia/Midnight-Miqu-70B-v1.0 --mem-fraction-static 0.9 --tp 8`
  
  
  ### Environment
  
  INFO 02-22 01:40:04 __init__.py:190] Automatically detected platform cuda.
  Python: 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]
  CUDA available: True
  GPU 0,1,2,3,4,5,6,7: NVIDIA H100 80GB HBM3
  GPU 0,1,2,3,4,5,6,7 Compute Capability: 9.0
  CUDA_HOME: /usr
  NVCC: Cuda compilation tools, release 12.4, V12.4.131
  CUDA Driver Version: 550.127.08
  PyTorch: 2.5.1+cu124
  sglang: 0.4.3.post2
  sgl_kernel: 0.0.3.post6
  flashinfer: 0.2.1.post2+cu124torch2.5
  triton: 3.1.0
  transformers: 4.48.3
  torchao: 0.8.0
  numpy: 1.26.4
  aiohttp: 3.11.12
  fastapi: 0.115.8
  hf_transfer: 0.1.9
  huggingface_hub: 0.29.1
  interegular: 0.3.3
  modelscope: 1.23.1
  orjson: 3.10.15
  packaging: 24.2
  psutil: 7.0.0
  pydantic: 2.10.6
  multipart: 0.0.20
  zmq: 26.2.1
  uvicorn: 0.34.0
  uvloop: 0.21.0
  vllm: 0.7.2
  openai: 1.63.2
  tiktoken: 0.9.0
  anthropic: 0.46.0
  decord: 0.6.0
  NVIDIA Topology:
          GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    NIC0    CPU Affinity    NUMA Affinity   GPU NUMA ID
  GPU0     X      NV18    NV18    NV18    NV18    NV18    NV18    NV18    SYS     0-103   0               N/A
  GPU1    NV18     X      NV18    NV18    NV18    NV18    NV18    NV18    SYS     0-103   0               N/A
  GPU2    NV18    NV18     X      NV18    NV18    NV18    NV18    NV18    SYS     0-103   0               N/A
  GPU3    NV18    NV18    NV18     X      NV18    NV18    NV18    NV18    SYS     0-103   0               N/A
  GPU4    NV18    NV18    NV18    NV18     X      NV18    NV18    NV18    SYS     104-207 1               N/A
  GPU5    NV18    NV18    NV18    NV18    NV18     X      NV18    NV18    SYS     104-207 1               N/A
  GPU6    NV18    NV18    NV18    NV18    NV18    NV18     X      NV18    SYS     104-207 1               N/A
  GPU7    NV18    NV18    NV18    NV18    NV18    NV18    NV18     X      SYS     104-207 1               N/A
  NIC0    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS      X
  
  Legend:
  
    X    = Self
    SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
    NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
    PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
    PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
    PIX  = Connection traversing at most a single PCIe bridge
    NV#  = Connection traversing a bonded set of # NVLinks
  
  NIC Legend:
  
    NIC0: mlx5_0
  
  
  Hypervisor vendor: KVM
  ulimit soft: 1024