AttributeError: '_OpNamespace' '_C' object has no attribute 'awq_marlin_repack'
#1
by
djdeniro
- opened
Got a error on -tp 1 --pp 6 / and -tp 2 -pp3 on 6x7900xtx
Loading safetensors checkpoint shards: 100% 25/25 [00:26<00:00, 1.06s/it]
vllm-1 | (VllmWorker rank=0 pid=417) INFO 07-23 08:19:18 [default_loader.py:262] Loading weights took 26.60 seconds
vllm-1 | (VllmWorker rank=1 pid=418) INFO 07-23 08:19:18 [default_loader.py:262] Loading weights took 26.73 seconds
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] WorkerProc failed to start.
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] Traceback (most recent call last):
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 485, in worker_main
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] worker = WorkerProc(*args, **kwargs)
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 382, in __init__
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] self.worker.load_model()
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 195, in load_model
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] self.model_runner.load_model(eep_scale_up=eep_scale_up)
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1827, in load_model
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] self.model = model_loader.load_model(
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] ^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] process_weights_after_loading(model, model_config, target_device)
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 115, in process_weights_after_loading
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] quant_method.process_weights_after_loading(module)
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/layers/quantization/awq_marlin.py", line 418, in process_weights_after_loading
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] marlin_w13_qweight = ops.awq_marlin_moe_repack(
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] ^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/vllm/_custom_ops.py", line 1038, in awq_marlin_moe_repack
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] output[e] = torch.ops._C.awq_marlin_repack(b_q_weight[e], size_k,
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1267, in __getattr__
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] raise AttributeError(
vllm-1 | (VllmWorker rank=0 pid=417) ERROR 07-23 08:19:18 [multiproc_executor.py:511] AttributeError: '_OpNamespace' '_C' object has no attribute 'awq_marlin_repack'
vllm-1 | [rank0]:[W723 08:19:19.737956138 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Doesn’t look like amd supports awq marlin or your build is missing it. In any case, I didn’t create this quantization.