RuntimeError: Can not found rewrite for auto_map: SkyworkChatModel
#1
by
xldistance
- opened
use command:lmdeploy serve api_server E:\models\Skywork-R1V3-38B-AWQ --server-port 5001 --session-len 70000 --max-batch-size 2048
And which awq model to use chat-template
I fixed the problem, inside config.json I have to change it to "AutoModel": "modeling_skywork_chat.InternVLChatModel",
"AutoModelForCausalLM": "modeling_skywork_chat. InternVLChatModel".
The following awq api command can be used.
lmdeploy serve api_server E:\models\Skywork-R1V3-38B-AWQ --server-port 5001 --session-len 70000 --chat-template qwq --reasoning-parser qwen-qwq