Update README.md

README.md

@@ -8,7 +8,9 @@ tags:
 
 
 Greetings! I've decided to share some insight that I've accumulated over the few years I've been toying around with LLMs, and the intricacies of how to potentially make them run better for creative writing or roleplay as the focus, but it might also help with technical jobs too.
+
 These might not be applicable with every model or user case, nor would it guarantee the best possible response with every single swipe, but it should help increase the odds of getting better mileage out of your model and experience, even if slightly, and help you avoid some bad or misled advice, which I personally have had to put up with. Some of this will be retreading old ground if you are already privy, but I will try to include less obvious stuff as well. Remember, I still consider myself a novice in some areas, and am always open to improvement.
+
 This list will probably be updated periodically.
 
 # 1: Instruct Formatting:
@@ -16,17 +18,21 @@ This list will probably be updated periodically.
 ## 1-1 What is the Instruct Template?
 
 The Instruct Template/Format is probably the most important when it comes to getting a model to work properly, as it is what encloses the training data with tokens that were used for the model, and your chat with said model. Some of them are used in a more general sense and are not brand specific, such as ChatML or Alpaca, while others stick to said brand, like Llama3 Instruct or Mistral Instruct. However not all models that are brand specific with their formatting will be trained with their own personal template.
+
 Its important to find out what format/template a model uses before booting it up, and you can usually check to see which it is on the model page. If a format isn't directly listed on said page, then there is ways to check internally with the local files. Each model has a tokenizer_config file, and sometimes even a special_tokens file, inside the main folder. As an example of what to look for, If you see something like a Mistral brand model that has im_start/im_end inside those files, then chances are that the person who finetuned it used ChatML tokens in their training data. Familiarizing yourself with the popular tokens used in training will help you navigate models better internally, especially if a creator forgets to post a readme on how it's suppose to function.
 
 ### 1-2 Is there any reason not to use the prescribed format/template?
 
 Sticking to the prescribed format will give your model better odds of getting things correct, or even better prose quality. But there are *some* small benefits when straying from the model's original format, such as supposedly being less censored. However the trade-off when it comes to maximizing a model's intelligence is never really worth it, and there are better ways to get uncensored responses with better prompting, or even tricking the model by editing their response slightly and continuing from there.
+
 From what I've found when testing models, if someone finetunes a model over the company's official Instruct focused model, instead of a base model, and doesn't use the underlining format that it was made with (such as ChatML over Mistral's 22B model as an example) then performance dips will kick in, giving less optimal responses then if it was instead using a unified format.
+
 This does not factor other occurrences of poor performance or context degradation when choosing to train on top of official Instruct models which may occur, but if it uses the correct format, and/or is trained with DPO or one of its variance (this one is more anecdotal, but DPO/ORPO/Whatever-O seems moreto be a more stable method when it comes to training on top of per-existing Instruct models) then the model will perform better overall.
 
 ### 1-3 What about models that list multiple formats/templates?
 
 This one is more due to model merging or choosing to forgo an Instruct model's format in training, although some people will choose to train their models like this, for whatever reason. In such an instance, you kinda just have to pick one and see what works best, but the merging of formats, and possibly even models, might provide interesting results, but only if its agreeable with the clutter on how you prompt it yourself. What do I mean by this? Well, perhaps its better if I give you a couple anecdotes on how this might work in practice...
+
 Nous-Capybara-limarpv3-34B is an older model at this point, but it has a unique feature that many models don't seem to implement; a Message Length Modifier. By adding small/medium/long at the end of the Assistant's Message Prefix, it will allow you to control how long the Bot's response is, which can be useful in curbing rambling, or enforcing more detail. Since Capybara, the underling model, uses the Vicuna format, its prompt typically looks like this:
 
 ```
@@ -60,6 +66,7 @@ The above example with Vicuna doesn't seem to work. However, by adding triple ha
 ```
 
 This is an example of where both formats can work together in a meaningful way.
+
 Another example is merging a Vicuna model with a ChatML one and incorporating the stop tokens from it, like with RP-Stew-v4. For reference, ChatML looks like this:
 
 ```
@@ -84,4 +91,5 @@ Then it will actually help prevent RP-Stew from rambling or repeating itself wit
 ### 1-4 Are stop tokens necessary?
 
 In my opinion, models work best when it has stop tokens built into them. Like with RP-Stew, the decrease in repetitive message length was about 25~33% on average, give or take from what I remember, when these end tokens are added. That's one case where the usefulness is obvious. Formats that use stop tokens tend to be more stable on average when it comes to creative back-and-forths with the bot, since it gives it a structure that's easier for it to understand when to end things, and inform better on who is talking.
+
 If you like your models to be unhinged and ramble on forever (aka; bad) then by all means, experiment by not using them. It might surprise you if you tweak it. But as like before, the intelligence hit is usually never worth it. Remember to make separate instances when experimenting with prompts, or be sure to put your tokens back in their original place. Otherwise you might end up with something dumb, like putting the stop token before the User in the User prefix.