xzuyn's picture

xzuyn

xzuyn

AI & ML interests

Doing stuff related to KoboldCPP

Recent Activity

Organizations

Caldera AI's profile picture xzuyn's Jar of Peanut Butter's profile picture Peanut Jar Mixers's profile picture BeaverAI's profile picture Peanut Jar Mixers Development's profile picture PJMixers Archive's profile picture Peanut Jar Mixers Images's profile picture

xzuyn's activity

replied to davidberenstein1957's post about 3 hours ago
view reply

You should be using <|begin▁of▁sentence|><|User|> instead of <|begin▁of▁sentence|>User:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B")
chat = [
  {"role": "user", "content": "EXAMPLE USER TURN 1"},
  {"role": "assistant", "content": "EXAMPLE MODEL TURN 1"},
  {"role": "user", "content": "EXAMPLE USER TURN 2"},
]

print(tokenizer.apply_chat_template(chat, add_generation_prompt=True, tokenize=False))
# <|begin▁of▁sentence|><|User|>EXAMPLE USER TURN 1<|Assistant|>EXAMPLE MODEL TURN 1<|end▁of▁sentence|><|User|>EXAMPLE USER TURN 2<|Assistant|>