tools calling
Hi, thank you for the GGUF model!
It seems there is an issue somewhere compare to 3.1 version with TOOL calling.
Test prompt:
read init.sh
Response:
<file contents> and relevant summary about it.
This is an existing file. It used to work with 3.1:
hf.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF:Q5_K_L
With 3.2 it does not work:
hf.co/bartowski/mistralai_Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q6_K
Test prompt:
read init.sh
Response:
[TOOL_CALLS]builtin_read_file<SPECIAL_32>{"filepath": "init.sh"}
Is this model config related issue or should I raise an issue with extension developer?
I've double checked model system prompts they are the same.
Extension is ContinueDev for VScode.
cross-discussion in Discord:
https://discord.com/channels/1108621136150929458/1385904830198845541
They did change something with tool calling and I tried to follow along but may not have gotten it perfect D:
@bartowski
thank you for reply!
Do you planning to have a look or its out of scope? It’s all magic to me, otherwise I would offer to help.
The biggest issue at this time is not being sure if the template is wrong or if the tool calling tools need updating to support it, since they explicitly called out in the model update a new template that works better for their model
Hey
@bartowski
, I've just checked version from unsloth - it seems to work fine with tools. I guess its open-sourced as well, so the fix is somewhere out there:hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q6_K