Model automatically loads with Alpaca. What is the correct template in Text generation webui?
#2
by
itsnottme
- opened
Hey, when I load the model in Text generation webui, it chooses Alpaca instruct template automatically, instead of the correct template.
Alpaca actually makes the model more creative but it's very likely to repeat phrases or generate gibberish.
I can't find a correct template for it. I see that readme says to use "Llama3", but the one below actually completely ruins how the model performs.
https://github.com/nickpotafiy/text-generation-webui/blob/main/instruction-templates/Llama-v3.yaml
For anyone that also has this issue, use an exl2 quant and the correct template will automatically be set. Also, uncheck "skip special tokens".
itsnottme
changed discussion status to
closed