unsloth

/

Qwen3-235B-A22B-GGUF

like 27

Follow

Unsloth AI 5.11k

Getting around 2-3 t/s with llama.cpp. It works great! Insane progress.

Waiting for UD quants.

I'm getting an error with prompt template though

common_chat_templates_init: failed to parse chat template (defaulting to chatml): Expected value expression at row 18, column 30:
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
^
{%- set index = (messages|length - 1) - loop.index0 %}

srv init: initializing slots, n_slots = 1
slot init: id 0 | task -1 | new slot n_ctx_slot = 8192
main: model loaded
main: chat template, chat_template: {%- for message in messages -%}
{{- '<|im_start|>' + message.role + '
' + message.content + '<|im_end|>
' -}}
{%- endfor -%}
{%- if add_generation_prompt -%}
{{- '<|im_start|>assistant
' -}}
{%- endif -%},

Maybe it will be fixed if I update llama.cpp?