Cannot run with tensor parallel > 1. Might need padding like on Qwen2.5-72B?
😔
2
#2 opened about 1 month ago
by
OwenArli
I get errors trying to deploy this in vllm or sglang.
😔
👍
6
3
#1 opened about 1 month ago
by
getfit