ubergarm

/

Qwen3-235B-A22B-Instruct-2507-GGUF

like 10

Does ikllama not support the jinja flag? So no tool calls?

we had pr for tool calling i was testing that but i think two days back everythink gets nuked and unfortunatly i nuked myself for changinging nvme so no back :p

edit

https://github.com/Thireus/ik_llama.cpp

you can check this copy

@koushd

I don't think it has --jinja but it does have

         --chat-template JINJA_TEMPLATE
                                  set custom jinja chat template (default: template taken from model's metadata)
                                  only commonly used templates are accepted:
                                  https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template

As for tool calling, give it a try. Are you hitting the /chat api endpoint or the full completions and giving it your own templated text?

There is also some kind of tool-call wrapper thing that @mtcl was using for some models too. But yeah bummer the improvements to tool/function calling PR was not yet merged. Hopefully things will get back up and running smoothly soon.

Let us know what you find!

yeah, I tried tool calling without the jinja flag. said it didn't handle tools.

Try this wrapper over ik_llama for tool calling support.

https://github.com/Teachings/FastAgentAPI

here is a video walkthrough for it : https://www.youtube.com/watch?v=JGo9HfkzAmc&t=1s

@koushd

Now that ik_llama.cpp is back, the fellow working on tool calling has a branch and example how to use it if you'd like to test: https://github.com/ikawrakow/ik_llama.cpp/pull/628#issuecomment-3103627677

thanks!

it only supports Kimi k2 according to the docs.

@koushd just got compiled in and suggests working with kimi and qwen3's if you want to pull and rebuild and try: https://github.com/ikawrakow/ik_llama.cpp/pull/628