Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference

Template wrong?

#2
by Hrre54353543 - opened

You suggest "Llama-3-Instruct-Names but you will need to uncheck "System same as user"." but if I use that template it just produces 1 sentence long stuff and not even talking had to switch back to the Llama-3.3-T4 for it to actually make several sentences and talk.

Owner

Don't know what the Llama-3.3-T4 template is (it isn't a default ST template), but any Llama 3 template will work.

Currently working on making this model a bit more steerable and coherent, but that probably won't be the cause of one liner responses. Check your system prompt, samplers and a few different characters / scenarios to try narrow down the issue.

I used the same settings as you suggested, although left system blank because there isn't any llama-3 instruct names template for system and it just didn't want to create long response nor talk. the Llama-3.3-t4 template is this one https://huggingface.co/sleepdeprived3/Llama-3.3-T4/tree/main

I'll try a few more times

Owner

Yep, just had a look at the one you linked and that'll work fine.

Llama-3-Instruct-Names template specifically you need to go in and uncheck the "System same as user" setting so that it enables the system role. Then it can be used as normal, I don't know why ST defaults to having it off but it is what it is. Without a system prompt / role the model will definitely underperform, as all the training I've done had system prompts in mind.

Sign up or log in to comment