kv_cache_utils.py NotImplementedError
(VllmWorker rank=0 pid=293) INFO 06-11 07:57:30 [backends.py:173] Compiling a graph for general shape takes 4.19 s
(VllmWorker rank=4 pid=297) INFO 06-11 07:57:31 [backends.py:173] Compiling a graph for general shape takes 4.21 s
(VllmWorker rank=2 pid=295) INFO 06-11 07:57:31 [backends.py:173] Compiling a graph for general shape takes 4.22 s
(VllmWorker rank=1 pid=294) INFO 06-11 07:57:31 [backends.py:173] Compiling a graph for general shape takes 4.28 s
(VllmWorker rank=0 pid=293) INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 47.64 s in total
(VllmWorker rank=7 pid=300) (VllmWorker rank=1 pid=294) INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 47.19 s in total
INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 49.01 s in total
(VllmWorker rank=2 pid=295) INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 48.66 s in total
(VllmWorker rank=5 pid=298) (VllmWorker rank=6 pid=299) INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 47.69 s in total
INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 48.18 s in total
(VllmWorker rank=3 pid=296) (VllmWorker rank=4 pid=297) INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 47.40 s in total
INFO 06-11 07:57:31 [monitor.py:34] torch.compile takes 47.51 s in total
ERROR 06-11 07:57:37 [core.py:515] EngineCore failed to start.
ERROR 06-11 07:57:37 [core.py:515] Traceback (most recent call last):
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 506, in run_engine_core
ERROR 06-11 07:57:37 [core.py:515] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 390, in init
ERROR 06-11 07:57:37 [core.py:515] super().init(vllm_config, executor_class, log_stats,
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 83, in init
ERROR 06-11 07:57:37 [core.py:515] self._initialize_kv_caches(vllm_config)
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 145, in _initialize_kv_caches
ERROR 06-11 07:57:37 [core.py:515] kv_cache_configs = [
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 146, in
ERROR 06-11 07:57:37 [core.py:515] get_kv_cache_config(vllm_config, kv_cache_spec_one_worker,
ERROR 06-11 07:57:37 [core.py:515] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/core/kv_cache_utils.py", line 960, in get_kv_cache_config
ERROR 06-11 07:57:37 [core.py:515] raise NotImplementedError
ERROR 06-11 07:57:37 [core.py:515] NotImplementedError