add regular tokenizer

by michaelfeil - opened May 21

←

May 21

No description provided.

15 days ago

It doesn't work for me with this tokenizer. The model loads, but the responses are a complete mess.

12 days ago

@peper1991 you need to apply the chat template yourself.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Cannot merge

This branch has merge conflicts in the following files:

· Sign up or log in to comment