A pure C++ OpenAI LLM service powered by TensorRT-LLM and GRPS, with support for Qwen3ForCausalLM.
#7
by
zhaocc1106
- opened
grps-trtllm have supported Qwen3ForCausalLM. Can give it a try if you are interested.
https://github.com/NetEase-Media/grps_trtllm/blob/master/docs%2Fqwen3.md
Support function call and enable_thinking param also.
zhaocc1106
changed discussion status to
closed