Bug in multi run function-call

#21

by judycc - opened 7 days ago

7 days ago

i localhost a kimi-k2-instruct ; and it's normal for only one function tool calling;
but when i just input an instruction for example "create a 2048 web game"; it broke down after just 2 steps , which is contrary to it should give a plan then execute step by step ;
i wonder there's bug in kimi or there's way to solve this problem
it's should be able to execute multi function tool call smoothly rather than interrupt with no reason

bigmoyan

Moonshot AI org 7 days ago

Hi, @judycc , did you update tokenizer_config.json and tokenization_kimi.py? We just fixed a multi-turn tool calls bug yesterday.

CaryH

7 days ago

This comment has been hidden (marked as Resolved)

judycc

7 days ago

Hi, @judycc , did you update tokenizer_config.json and tokenization_kimi.py? We just fixed a multi-turn tool calls bug yesterday.

yes, i've already updated these files, maybe the version of sglang make some mistakes (although i used sglang-0.4.9.post2)

doramonk

6 days ago

@bigmoyan pls share which sglang and vllm build that is working with K2. seems that the default vllm/sglang does not work well.

youkaichao

Moonshot AI org 6 days ago

@doramonk you need to use the nightly vllm, the kimi tool parser is not in the release yet. see https://x.com/vllm_project/status/1943688429205860584

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment