Add full tool calling support to chat template
#20
by
CISCai
- opened
This adds full tool calling support to the chat template, producing pretty much identical results as the the prompt processing code in tokenization_minicpm.py. Fully supports generating tool_call (requires arguments to be a dict (as supported in recent transformers) and not a string) and thought history.
Also sets add_bos_token to false to prevent <s> from being added to the prompt when not using the chat template, as it clearly should not be.
Thanks for your contribution. It works for most cases, but there seem to be a few minor problems:
- Some parameters that are supported in the JSON schema, such as "default" and "pattern," are not supported in the chat template.
example code [CLICK TO EXPAND]
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM3-4B", trust_remote_code=True)
messages = [{"role": "user", "content": "What is the weather like in Boston?."}]
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"date": {
"type": "string",
"description": "The date for which to get the weather, e.g. 2022-01-01",
"pattern": "^([1-9] |1[0-9]| 2[0-9]|3[0-1])(.|-)([1-9] |1[0-2])(.|-|)20[0-9][0-9]$",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"default": "celcius",
},
},
"required": ["location"],
},
},
}
]
prompt = tokenizer.apply_chat_template(
messages, tools=tools, tokenize=False, add_generation_start=True
)
print(prompt)
expected prompt [CLICK TO EXPAND]
<|im_start|>system
# Functions
Here is a list of functions that you can invoke:
```python
from enum import Enum
from typing import List, Dict, Optional
from pydantic import BaseModel, Field
class Unit(Enum):
celsius = 'celsius'
fahrenheit = 'fahrenheit'
def get_current_weather(location, date=None, unit='celcius'):
"""
Get the current weather in a given location
Args:
location: str = Field(..., description='The city and state, e.g. San Francisco, CA')
date: Optional[str] = Field(
None,
description='The date for which to get the weather, e.g. 2022-01-01',
regex='^([1-9] |1[0-9]| 2[0-9]|3[0-1])(.|-)([1-9] |1[0-2])(.|-|)20[0-9][0-9]$',
)
unit: Optional[Unit] = 'celcius'
"""
```
# Function Call Rule and Output Format
- If the user's question can be answered without calling any function, please answer the user's question directly. In this situation, you should return your thought and answer the user's question directly.
- If the user cannot be answered without calling any function, and the user does not provide enough information to call functions, please ask the user for more information. In this situation, you should return your thought and ask the user for more information.
- If the user's question cannot be answered without calling any function, and the user has provided enough information to call functions to solve it, you should call the functions. In this situation, the assistant should return your thought and call the functions.
- Use default parameters unless the user has specified otherwise.
- You should answer in the following format:
<|thought_start|>
{explain why the user's question can be answered without calling a function or why you should ask the user for more information or why you should call one or more functions and your plan to solve the user's question.}
<|thought_end|>
<|tool_call_start|>
```python
func1(params_name=params_value, params_name2=params_value2...)
func2(params)
```
<|tool_call_end|>
{answer the user's question directly or ask the user for more information}<|im_end|>
<|im_start|>user
What is the weather like in Boston?.<|im_end|>
- The chat template will add a <|im_end|> token at the beginning of the prompt for simple conversations without tools.
example code [CLICK TO EXPAND]
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM3-4B", trust_remote_code=True)
messages = [{"role": "user", "content": "What is the weather like in Boston?."}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_start=True
)
print(prompt)
expected prompt [CLICK TO EXPAND]
<|im_start|>user
What is the weather like in Boston?.<|im_end|>
We are glad to receive your updates. Thanks for your contribution again.
Default is easy to add, not so sure about regex pattern though. :( ah, wait, now I see, should be simple.
Done, what do you think? :)
(Looking into your second point now)
For some complex paramters with such as object, the chat template may not include the detailed information into the system prompt.
object type parameter example [CLICK TO EXPAND]
#!/usr/bin/env python
# encoding: utf-8
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM3-4B", trust_remote_code=True)
messages = [{"role": "user", "content": "hello"}]
tools = [
{
"type": "function",
"function": {
"name": "create_user",
"description": "creates a user",
"parameters": {
"type": "object",
"properties": {
"user": {
"title": "User",
"type": "object",
"properties": {
"user_id": {
"title": "User Id",
"description": "The unique identifier for a user",
"default": 0,
"type": "integer",
},
"name": {
"title": "Name",
"description": "The name of the user",
"type": "string",
},
"age": {
"title": "Age",
"description": "The age of the user",
"default": 25,
"type": "integer",
},
"email": {
"title": "Email",
"description": "The email address of the user",
"type": "string",
},
"friends": {
"title": "Friends",
"description": "List of friends of the user",
"type": "array",
"items": {"type": "string"},
},
},
"required": ["name", "email"],
},
},
"required": ["user"],
"definitions": {},
},
},
},
]
prompt = tokenizer.apply_chat_template(
messages, tools=tools, tokenize=False, add_generation_start=True
)
print(prompt)
expected prompt [CLICK TO EXPAND]
# Functions
Here is a list of functions that you can invoke:
```python
from enum import Enum
from typing import List, Dict, Optional
from pydantic import BaseModel, Field
class User(BaseModel):
user_id: Optional[int] = Field(
0, description='The unique identifier for a user', title='User Id'
)
name: str = Field(..., description='The name of the user', title='Name')
age: Optional[int] = Field(25, description='The age of the user', title='Age')
email: str = Field(..., description='The email address of the user', title='Email')
friends: Optional[List[str]] = Field(
None, description='List of friends of the user', title='Friends'
)
def create_user(user):
"""
creates a user
Args:
user: User = Field(..., title='User')
"""
```
# Function Call Rule and Output Format
- If the user's question can be answered without calling any function, please answer the user's question directly. In this situation, you should return your thought and answer the user's question directly.
- If the user cannot be answered without calling any function, and the user does not provide enough information to call functions, please ask the user for more information. In this situation, you should return your thought and ask the user for more information.
- If the user's question cannot be answered without calling any function, and the user has provided enough information to call functions to solve it, you should call the functions. In this situation, the assistant should return your thought and call the functions.
- Use default parameters unless the user has specified otherwise.
- You should answer in the following format:
<|thought_start|>
{explain why the user's question can be answered without calling a function or why you should ask the user for more information or why you should call one or more functions and your plan to solve the user's question.}
<|thought_end|>
<|tool_call_start|>
```python
func1(params_name=params_value, params_name2=params_value2...)
func2(params)
```
<|tool_call_end|>
{answer the user's question directly or ask the user for more information}<|im_end|>
<|im_start|>user
hello<|im_end|>
Ah, that's when it uses BaseModel, I was wondering about that, cool, I'll look into it. :)