RedHatAI/Kimi-K2-Instruct-quantized.w4a16 · Keep outputting empty content without stopping

8 days ago

ChatCompletionChunk(id='chatcmpl-185e77b2876e457b99d913210ca20b1d', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason='length', index=0, logprobs=None, stop_reason=None)], created=1752815639, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)

I found that after deploying with vllm, it kept outputting content= "" without stopping until finish_reason='length'.

zhnagchenchne

7 days ago

ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)
ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)
ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)
ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)
ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)
ChatCompletionChunk(id='chatcmpl-3c7d2bf8b8634e8594c2254c0a3583fc', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason='length', index=0, logprobs=None, stop_reason=None)], created=1752830061, model='kimi-k2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)

ekurtic

Red Hat AI org 4 days ago

Hey, thanks for reporting. I've answered your question here: https://github.com/vllm-project/vllm/issues/21181

ekurtic changed discussion status to closed 4 days ago