--- title: 支持的工具列表 & llm返回的工具调用 --- - [tools调用](#tools调用) - [LLM入参 = messages + tools](#llm入参--messages--tools) - [LLM出参 = 工具名 + 参数](#llm出参--工具名--参数) - [LLM入参示例：messages and tools](#llm入参示例messages-and-tools) - [LLM入参示例：转化为 prompt 字符串](#llm入参示例转化为-prompt-字符串) - [llama3.1-405b ⭐️⭐️](#llama31-405b-️️) - [Hermes-3-Llama-3.1-405B ⭐️⭐️⭐️](#hermes-3-llama-31-405b-️️️) - [llama4 ⭐️⭐️⭐️](#llama4-️️️) - [mistralai/Ministral-8B-Instruct-2410](#mistralaiministral-8b-instruct-2410) - [qwen3 ⭐️⭐️⭐️⭐️⭐️](#qwen3-️️️️️) - [deepseek-r1 ⭐️⭐️](#deepseek-r1-️️) - [deepseek-v3.1 ⭐️⭐️](#deepseek-v31-️️) - [gemma-3-27b-it ⭐️](#gemma-3-27b-it-️) - [grok-2 ⭐️](#grok-2-️) ## tools调用 ```py # openai chat api completion = client.chat.completions.create( model=model_name, messages=messages, tools=tools, # tool list defined above tool_choice="auto" ) # open response api ``` ## LLM入参 = messages + tools tools是支持的工具列表 一般都是json格式，也有非json格式 - json格式: s - 非json格式: s ## LLM出参 = 工具名 + 参数 llm返回格式通常有两种，`pythonic` 和 `json` 格式 json response 示例 [llama3.2_json](https://github.com/vllm-project/vllm/blob/v0.10.1/examples/tool_chat_template_llama3.2_json.jinja) ```sh please respond with JSON for a function call. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. ``` pythonic response 示例 [llama3.2_pythonic 非官方](https://github.com/vllm-project/vllm/blob/v0.10.1/examples/tool_chat_template_llama3.2_pythonic.jinja) ```sh Please respond with a python list of the calls. Respond in the format [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)] ``` 多数都采用的json返回格式。 ## LLM入参示例：messages and tools ```py messages = [ { "role": "system", "content": "You are a bot that responds to weather queries." }, { "role": "user", "content": "Hey, what's the temperature in Paris right now?" } ] # chat_completion = client.chat.completions.create(messages=messages, model=model, tools=tools) # 可以这样调用 LLM ``` `json_schema` of tools ```json { "type": "function", "function": { "name": "get_current_temperature", "description": "Get the current temperature at a location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the temperature for, in the format \"City, Country\"" } }, "required": [ "location" ] }, "return": { "type": "number", "description": "The current temperature at the specified location in the specified units, as a float." } } } ``` ## LLM入参示例：转化为 prompt 字符串 ### llama3.1-405b ⭐️⭐️ ```sh <|begin_of_text|><|start_header_id|>system<|end_header_id|> Environment: ipython Cutting Knowledge Date: December 2023 Today Date: 26 Jul 2024 You are a bot that responds to weather queries.<|eot_id|><|start_header_id|>user<|end_header_id|> Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables. { "type": "function", "function": { "name": "get_current_temperature", "description": "Get the current temperature at a location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the temperature for, in the format \"City, Country\"" } }, "required": [ "location" ] }, "return": { "type": "number", "description": "The current temperature at the specified location in the specified units, as a float." } } } Hey, what's the temperature in Paris right now?<|eot_id|><|start_header_id|>assistant<|end_header_id|> ``` - **入参**: - **tools格式**: 支持的工具列表（`tools`） 是基于 json-schema 的，因为chat_template中采用的是 [`tojson(indent=4)`](https://github.com/vllm-project/vllm/blob/v0.10.1/examples/tool_chat_template_llama3.1_json.jinja#L48) - **tools在prompt中的位置**: 拼在第一个user轮内容的前面。(system不变，其他轮message也不变) - **出参**: - **出参格式**: 返回的`respone` 要求是 json 格式，`respond with a JSON ... Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}` - **出参的工具解析**: ? - **评价**： - 扣分项: - 入参: `Environment: ipython` - 入参: `Today Date` - 出参格式: 返回内容没有 `` 包裹，不方便结果的解析。 - **推荐指数**: ⭐️⭐️ ### Hermes-3-Llama-3.1-405B ⭐️⭐️⭐️ ```sh <|begin_of_text|><|im_start|>system You are a function calling AI model. You are provided with function signatures within XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: {"type": "function", "function": {"name": "get_current_temperature", "description": "get_current_temperature(location: str) -> float - Get the current temperature at a location. Args: location(str): The location to get the temperature for, in the format "City, Country" Returns: The current temperature at the specified location in the specified units, as a float.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, Country\""}}, "required": ["location"]}} Use the following pydantic model json schema for each tool call you will make: {"properties": {"name": {"title": "Name", "type": "string"}, "arguments": {"title": "Arguments", "type": "object"}}, "required": ["name", "arguments"], "title": "FunctionCall", "type": "object"}} For each function call return a json object with function name and arguments within XML tags as follows: {"name": , "arguments": } <|im_end|> <|im_start|>system You are a bot that responds to weather queries.<|im_end|> <|im_start|>user Hey, what's the temperature in Paris right now?<|im_end|> <|im_start|>assistant ``` > 为方便查看，以上prompt手动变化了缩进。 - **入参**: - **tools格式**: 支持的工具列表（`tools`） 是基于自定义格式的（类json-schema），详见 [chat_template](https://huggingface.co/spaces/xu-song/tokenizer-arena/blob/main/docs/chat-template/Hermes-3-Llama-3.1-405B/chat_template.tool_use.jinja#L42) - **tools在prompt中的位置**: 额外增加了一个`system`轮，放在最前面。（用户设置的`system`保持不变），这个额外的system轮告诉大模型可以使用tools中的工具，并且指定返回格式 - **出参**: 返回的`respone` 要求是 `` 包裹的json `return a json object with function name and arguments within XML tags as follows: {"name": , "arguments": }` - **评价**: - 扣分项: 入参的`tools格式`与json-schema的字段虽然一样，也类似json，但不是严格的json格式，通用性差。 - **推荐指数**: ⭐️⭐️⭐️ ### llama4 ⭐️⭐️⭐️ ```sh <|begin_of_text|><|header_start|>system<|header_end|> Environment: ipython You are a bot that responds to weather queries.<|eot|><|header_start|>user<|header_end|> Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables. { "type": "function", "function": { "name": "get_current_temperature", "description": "Get the current temperature at a location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the temperature for, in the format \"City, Country\"" } }, "required": [ "location" ] }, "return": { "type": "number", "description": "The current temperature at the specified location in the specified units, as a float." } } } Hey, what's the temperature in Paris right now?<|eot|><|header_start|>assistant<|header_end|> ``` 跟 llama3.1差不多，只是少了`date`，并且换了`special_token`。(同样拼在第一个user轮) - **评价**: 同 llama3.1 - **推荐指数**: ⭐️⭐️⭐️ ### mistralai/Ministral-8B-Instruct-2410 ```sh [AVAILABLE_TOOLS][{"type": "function", "function": {"name": "get_current_temperature", "description": "Get the current temperature at a location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, Country\""}}, "required": ["location"]}}}][/AVAILABLE_TOOLS][INST]You are a bot that responds to weather queries. Hey, what's the temperature in Paris right now?[/INST] ``` - **入参**: 支持的工具列表（`tools`） 是基于 的， - **出参**: 返回的`respone` 要求是 ### qwen3 ⭐️⭐️⭐️⭐️⭐️ ```sh <|im_start|>system You are a bot that responds to weather queries. # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within XML tags: {"type": "function", "function": {"name": "get_current_temperature", "description": "Get the current temperature at a location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The location to get the temperature for, in the format \"City, Country\""}}, "required": ["location"]}, "return": {"type": "number", "description": "The current temperature at the specified location in the specified units, as a float."}}} For each function call, return a json object with function name and arguments within XML tags: {"name": , "arguments": } <|im_end|> <|im_start|>user Hey, what's the temperature in Paris right now?<|im_end|> <|im_start|>assistant ``` - **入参**: - **tools格式**: 支持的工具列表（`tools`） 是基于 json-schema 的. - **tools在prompt中的位置**: 拼接到原始`system`的结尾 + 额外的prompt (告诉大模型可以使用tools中的工具，并且指定返回格式) - **出参**: 返回的`respone` 要求是 `` 包裹的json `return a json object with function name and arguments within XML tags:\n\n{"name": , "arguments": }\n` - **推荐指数**: ⭐️⭐️⭐️⭐️⭐️ ### deepseek-r1 ⭐️⭐️ ```sh <｜begin▁of▁sentence｜>You are a bot that responds to weather queries.<｜User｜>Hey, what's the temperature in Paris right now?<｜Assistant｜> ``` - **评价**: - 扣分项: 不支持tools，tool_calls也有问题 - **推荐指数**: ⭐️⭐️ ### deepseek-v3.1 ⭐️⭐️ 与 deepseek-r1 一样 ### gemma-3-27b-it ⭐️ 不支持任何tool ### grok-2 ⭐️ 不支持任何tool