John6666/text2tag-llm · Custom Format for Mistral

Oct 25

•

When i saw the llama-cpp-agent, the mistral prompt makers is looks like combining v1 and tekken chat template.
Based on Mistral Cookbook and inflatebot's SillyTavern template
i generate this code with LLM

from llama_cpp_agent.messages_formatter import MessagesFormatter, PromptMarkers, Roles
mistral_v1_markers = {
    Roles.system: PromptMarkers(""" [INST] """, """ [/INST] Understood.</s>"""),
    Roles.user: PromptMarkers(""" [INST] """, """ [/INST]"""),
    Roles.assistant: PromptMarkers(" ", "</s>"),
    Roles.tool: PromptMarkers("", ""),
}
mistral_v2_markers = {
    Roles.system: PromptMarkers("""[INST] """, """[/INST] Understood.</s>"""),
    Roles.user: PromptMarkers("""[INST] """, """[/INST]"""),
    Roles.assistant: PromptMarkers(" ", "</s>"),
    Roles.tool: PromptMarkers("", ""),
}
mistral_v3_tekken_markers = {
    Roles.system: PromptMarkers("""[INST]""", """[/INST]Understood.</s>"""),
    Roles.user: PromptMarkers("""[INST]""", """[/INST]"""),
    Roles.assistant: PromptMarkers("", "</s>"),
    Roles.tool: PromptMarkers("", ""),
}

mistral_v1_formatter = MessagesFormatter(
    pre_prompt="",
    prompt_markers=mistral_v1_markers,
    include_sys_prompt_in_first_user_message=False,
    default_stop_sequences=["</s>"]
)
mistral_v2_formatter = MessagesFormatter(
    pre_prompt="",
    prompt_markers=mistral_v2_markers,
    include_sys_prompt_in_first_user_message=False,
    default_stop_sequences=["</s>"]
)
mistral_v3_tekken_formatter = MessagesFormatter(
    pre_prompt="",
    prompt_markers=mistral_v3_tekken_markers,
    include_sys_prompt_in_first_user_message=False,
    default_stop_sequences=["</s>"]
)

i haven't tested it but maybe it's useful

John6666

Owner Oct 25

Thanks for the contribution! I made a bit of a mistake at first, but your part of the code seems to work just fine as is. This change also makes it easier to add templates in the future.

I read the code on github when I bugged it, and it seems that the preset templates are not that complex to define. We can customize it without worry.
https://github.com/Maximilian-Winter/llama-cpp-agent/blob/master/src/llama_cpp_agent/messages_formatter.py