Benefit of official chat-template
#19
by
Arpit-Bansal
- opened
Has anyone noticed or tested any performance enhancement, if use ths officila chat-template:
{{ bos_token }}{% for message in messages %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}
instead of :
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"""content"""}
]