Repetitiveness
Maybe its because I’m using a 4.0 BPW but I’ve noticed this model likes to repeat words. For example, when introducing technology to humanity in a roleplay scenario it likes to throw out “Game-Changer” and such words, without really going into detail or describing anything.
I can ban them in silly tavern, but I’m curious on if this is an issue with the dataset.
I have not tried the grey model, which is on my to-do list.
I really do enjoy this model, though.
Thanks for the feedback, FrenzyBiscuit, and sorry for the long answering delay.
This model is quite a loaded merge of various models.. and merges. So issues might occur related to one or several models. The most problematic ones imo are Tulu (included in Smartracks), Doppel (based on Hermes, which can be tricky due to its tokenizer), and maybe Tess-3 that I use as a perplexity dropper and is prone to repeat itself (common issue with low ppl Llama 3 bases). V3 is out, but I doubt it changes anything much.
Grey is more classic, using one of my old Smatricks base (which was stable), and my usual favorite finetunes to color it. It's a solid model.