Chat template differs from OpenAI's. Is it expected?
#6
by
gentry1337
- opened
Repro:
from transformers import AutoTokenizer
chat = [
{"role": "system", "content": "Let's sing!"},
{"role": "user", "content": "Because maybe"},
{"role": "assistant", "content": "You're gonna be the one that saves me"},
{"role": "user", "content": "And after all"},
{"role": "assistant", "content": "You're my wonderwall"},
]
openai_tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")
unsloth_tokenizer = AutoTokenizer.from_pretrained("unsloth/gpt-oss-20b-BF16")
openai_result = openai_tokenizer.apply_chat_template(chat, tokenize=False)
unsloth_result = unsloth_tokenizer.apply_chat_template(chat, tokenize=False)
if openai_result != unsloth_result:
print("Results differ:")
print("OpenAI result: ", openai_result)
print("Unsloth result: ", unsloth_result)
Output:
Results differ:
OpenAI result: <|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-28
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions
Let's sing!
<|end|><|start|>user<|message|>Because maybe<|end|><|start|>assistant<|channel|>final<|message|>You're gonna be the one that saves me<|end|><|start|>user<|message|>And after all<|end|><|start|>assistant<|channel|>final<|message|>You're my wonderwall<|return|>
Unsloth result: <|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-28
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.
Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>developer<|message|># Instructions
Let's sing!<|end|><|start|>user<|message|>Because maybe<|end|><|start|>assistant<|message|>You're gonna be the one that saves me<|end|><|start|>user<|message|>And after all<|end|><|start|>assistant<|message|>You're my wonderwall<|return|>
I've also run this test for unsloth 20b vs unsloth 120b and openai 20b vs openai 120b - unsloth tokenization even differs between 20B and 120B tokenizers while OpenAI's matches between different model sizes.
Looks like I've found some explanations of difference of chat templates:
https://docs.unsloth.ai/basics/gpt-oss-how-to-run-and-fine-tune#unsloth-fixes-for-gpt-oss
While this indeed explains the difference between OpenAI's chat template and Unsloth's one, it doesn't explain the difference between Unsloth 20B and Unsloth 120B GPT-OSS:
Results differ:
Unsloth 20B result: <|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-28
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.
Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>developer<|message|># Instructions
Let's sing!<|end|><|start|>user<|message|>Because maybe<|end|><|start|>assistant<|message|>You're gonna be the one that saves me<|end|><|start|>user<|message|>And after all<|end|><|start|>assistant<|message|>You're my wonderwall<|return|>
Unsloth 120B result: <|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-28
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions
Let's sing!<|end|><|start|>user<|message|>Because maybe<|end|><|start|>assistant<|channel|>final<|message|>You're gonna be the one that saves me<|end|><|start|>user<|message|>And after all<|end|><|start|>assistant<|channel|>final<|message|>You're my wonderwall<|return|>
It seems like GPT-OSS 120B misses Calls to these tools must go to the commentary channel: instruction