unsloth

/

GLM-4.1V-9B-Thinking-GGUF

like 24

Follow

Unsloth AI 7.81k

is there any combination of "temp" parameters to stop responding without cease

Hello, did you use the jinja chat template?

yeap, I use this one

[gMASK]
{%- for msg in messages %}
{%- if msg.role == 'system' %}
<|system|>
{{ msg.content }}
{%- elif msg.role == 'user' %}
<|user|>{{ '\n' }}

    {%- if msg.content is string %}

{{ msg.content }}
{%- else %}
{%- for item in msg.content %}
{%- if item.type == 'video' or 'video' in item %}
<|begin_of_video|><|video|><|end_of_video|>
{%- elif item.type == 'image' or 'image' in item %}
<|begin_of_image|><|image|><|end_of_image|>
{%- elif item.type == 'text' %}
{{ item.text }}
{%- endif %}
{%- endfor %}
{%- endif %}
{%- elif msg.role == 'assistant' %}
{%- if msg.metadata %}
<|assistant|>{{ msg.metadata }}
{{ msg.content }}
{%- else %}
<|assistant|>
{{ msg.content }}
{%- endif %}
{%- endif %}
{%- endfor %}
{% if add_generation_prompt %}<|assistant|>
{% endif %}

GLM-4.1V is not supported by llama.cpp yet.
The format messed up EOS token.

https://github.com/ggml-org/llama.cpp/issues/14495