meta-llama

/

Meta-Llama-3.1-70B-Instruct

like 492

Hello!

When adding a response from a tool in string format to the list of messages to a model, for example:

{
    "messages": [
    ...
    {
        "role": "tool",
        "content": "Project 'AUTOTEST' is exists."
    }
]

the error returned is (in HF TGI 2.2.0):

{
    "error": "Template error: unknown filter: filter tojson is unknown (in <string>:100)",
    "error_type": "template_error"
}

The error is in this condition in chat_template:

        {%- if message.content is mapping or message.content is iterable %}
            {{- message.content | tojson }}

String in Python is iterable too.

Fix:

        {%- if message.content is mapping or (message.content is iterable and not message.content is string) %}
            {{- message.content | tojson }}

Please fix chat_template in the repository.

Thanks!