ParasiticRogue commited on
Commit
1d8ac53
·
verified ·
1 Parent(s): 9117988

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +212 -95
README.md CHANGED
@@ -7,33 +7,45 @@ tags:
7
  ---
8
 
9
 
10
- Greetings! I've decided to share some insight that I've accumulated over the few years I've been toying around with LLMs, and the intricacies of how to potentially make them run better for creative writing or roleplay as the focus, but it might also help with technical jobs too.
11
 
12
- These might not be applicable with every model or user case, nor would it guarantee the best possible response with every single swipe, but it should help increase the odds of getting better mileage out of your model and experience, even if slightly, and help you avoid some bad or misled advice, which I personally have had to put up with. Some of this will be retreading old ground if you are already privy, but I will try to include less obvious stuff as well. Remember, I still consider myself a novice in some areas, and am always open to improvement.
13
 
14
- This list will probably be updated periodically.
15
 
16
- # 1: Instruct Formatting:
17
 
18
- ## 1-1 What is the Instruct Template?
19
 
20
- The Instruct Template/Format is probably the most important aspect when it comes to getting a model to work properly, as it is what encloses the training data with tokens that were used for the model, and your chat with said model. Some of them are used in a more general sense and are not brand specific, such as ChatML or Alpaca, while others stick to said brand, like Llama3 Instruct or Mistral Instruct. However not all models that are brand specific with their formatting will be trained with their own personal template.
21
 
22
- Its important to find out what format/template a model uses before booting it up, and you can usually check to see which it is on the model page. If a format isn't directly listed on said page, then there is ways to check internally with the local files. Each model has a tokenizer_config file, and sometimes even a special_tokens file, inside the main folder. As an example of what to look for, If you see something like a Mistral brand model that has im_start/im_end inside those files, then chances are that the person who finetuned it used ChatML tokens in their training data. Familiarizing yourself with the popular tokens used in training will help you navigate models better internally, especially if a creator forgets to post a readme on how it's suppose to function.
23
 
24
- ### 1-2 Is there any reason not to use the prescribed format/template?
25
 
26
- Sticking to the prescribed format will give your model better odds of getting things correct, or even better prose quality. But there are *some* small benefits when straying from the model's original format, such as supposedly being less censored. However the trade-off when it comes to maximizing a model's intelligence is never really worth it, and there are better ways to get uncensored responses with better prompting, or even tricking the model by editing their response slightly and continuing from there.
27
 
28
- From what I've found when testing models, if someone finetunes a model over the company's official Instruct focused model, instead of a base model, and doesn't use the underlining format that it was made with (such as ChatML over Mistral's 22B model as an example) then performance dips will kick in, giving less optimal responses then if it was instead using a unified format.
29
 
30
- This does not factor other occurrences of poor performance or context degradation when choosing to train on top of official Instruct models which may occur, but if it uses the correct format, and/or is trained with DPO or one of its variance (this one is more anecdotal, but DPO/ORPO/Whatever-O seems to be a more stable method when it comes to training on top of per-existing Instruct models) then the model will perform better overall.
31
 
32
- ### 1-3 What about models that list multiple formats/templates?
33
 
34
- This one is more due to model merging or choosing to forgo an Instruct model's format in training, although some people will choose to train their models like this, for whatever reason. In such an instance, you kinda just have to pick one and see what works best, but the merging of formats, and possibly even models, might provide interesting results, but only if its agreeable with the clutter on how you prompt it yourself. What do I mean by this? Well, perhaps its better if I give you a couple anecdotes on how this might work in practice...
35
 
36
- Nous-Capybara-limarpv3-34B is an older model at this point, but it has a unique feature that many models don't seem to implement; a Message Length Modifier. By adding small/medium/long at the end of the Assistant's Message Prefix, it will allow you to control how long the Bot's response is, which can be useful in curbing rambling, or enforcing more detail. Since Capybara, the underling model, uses the Vicuna format, its prompt typically looks like this:
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
  ```
39
  System:
@@ -41,23 +53,27 @@ User:
41
  Assistant:
42
  ```
43
 
44
- Meanwhile, the limarpv3 lora, which has the Message Length Modifier, was used on top of Capybara and chose to use Alpaca as its format:
45
 
46
  ```
47
  ### Instruction:
48
  ### Input:
49
- ### Response: (length = short/medium/long/etc)
50
  ```
51
 
52
- Seems to be quite different, right? Well, it is, but we can also combine these two formats in a meaningful way and actually see tangible results. When using Nous-Capybara-limarpv3-34B with its underling Vicuna format and the Message Length Modifier together, the results don't come together, and you have basically 0 control on its length:
 
 
53
 
54
  ```
55
  System:
56
  User:
57
- Assistant: (length = short/medium/long/etc)
58
  ```
59
 
60
- The above example with Vicuna doesn't seem to work. However, by adding triple hashes to it, the modifier actually will take effect, making the messages shorter or longer on average depending on how you prompt it.
 
 
61
 
62
  ```
63
  ### System:
@@ -65,9 +81,13 @@ The above example with Vicuna doesn't seem to work. However, by adding triple ha
65
  ### Assistant: (length = short/medium/long/etc)
66
  ```
67
 
68
- This is an example of where both formats can work together in a meaningful way.
 
 
69
 
70
- Another example is merging a Vicuna model with a ChatML one and incorporating the stop tokens from it, like with RP-Stew-v4. For reference, ChatML looks like this:
 
 
71
 
72
  ```
73
  <|im_start|>system
@@ -78,7 +98,9 @@ User prompt<|im_end|>
78
  Bot response<|im_end|>
79
  ```
80
 
81
- One thing to note is that, unlike Alpaca, the ChatML template has System/User/Assistant inside it, making it vaguely similar to Vicuna. Vicuna itself doesn't have stop tokens, but if we add them like so:
 
 
82
 
83
  ```
84
  SYSTEM: system prompt<|end|>
@@ -86,148 +108,243 @@ USER: user prompt<|end|>
86
  ASSISTANT: assistant output<|end|>
87
  ```
88
 
89
- Then it will actually help prevent RP-Stew from rambling or repeating itself within the same message, and also lowering the chances of your bot speaking as the user. When merging models I find it best to keep to one format in order to keep its performance high, but there can be rare cases where mixing them could work.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
- ### 1-4 Are stop tokens necessary?
 
 
 
92
 
93
- In my opinion, models work best when it has stop tokens built into them. Like with RP-Stew, the decrease in repetitive message length was about 25~33% on average, give or take from what I remember, when these end tokens are added. That's one case where the usefulness is obvious. Formats that use stop tokens tend to be more stable on average when it comes to creative back-and-forths with the bot, since it gives it a structure that's easier for it to understand when to end things, and inform better on who is talking.
94
 
95
- If you like your models to be unhinged and ramble on forever (aka; bad) then by all means, experiment by not using them. It might surprise you if you tweak it. But as like before, the intelligence hit is usually never worth it. Remember to make separate instances when experimenting with prompts, or be sure to put your tokens back in their original place. Otherwise you might end up with something dumb, like putting the stop token before the User in the User prefix.
96
 
97
- # 2: Character/Chat Formatting
98
 
99
- ## 2-1 What is a Character Card?
100
 
101
- Lets get the obvious thing out of the way. Character Cards are basically personas of, well, characters, be it from real life, an established franchise, or someone's OC, for the AI bot to impersonate and interact with. The layout of a Character Card is typically written in the form of a profile or portfolio, with different styles available for approaching the technical aspects of listing out what makes them unique.
102
 
103
- ### 2-2 What are the different styles of Character Cards?
104
 
105
- Making a card isn't exactly a solved science, and the way its prompted could vary the outcome between different model brands and model sizes. However, there are a few that are popular among the community that have gained traction.
 
 
 
106
 
107
- One way to approach it is a simply writing out the character's persona like you would in a novel/book, using natural prose to describe their background and appearance. Though this method would require a deft hand/mind to make sure it flows well and doesn't repeat too much with specific keywords, and might be a bit harder compered to some of the other styles if you are just starting out. More useful for pure writers, probably.
 
 
 
108
 
109
- Another is doing a list format, where every feature is placed out categorically and sufficiently. There are different ways of doing this as well, like markdown, wiki style, or the community made W++, just to name a few.
 
 
 
110
 
111
- Some use parentheses or brackets to enclose each section, some use dashes for separate listings, some bold sections with hashes or double asterisks, or some none of the above.
112
 
113
- I haven't found which one is objectively the best when it comes to a specific format, although W++ is probably the worst of the bunch when it comes to stabilization, with Wiki Style taking second worse just because of it being bloat dumped from said wiki. There could be a myriad of reasons why W++ might not be considered as much anymore, but my best guess is, since the format is non-standard in most model's training data, it has less to pull from in its reasoning.
 
 
114
 
115
- My current recommendation is just to use some mixture of lists and regular prose, with a traditional list when it comes to appearance and traits, and using normal writing for background and speech. Though you should be mindful of what perspective you prompt the card beforehand.
116
 
117
- ### 2-3 What writing perspectives should I consider before making a card?
118
 
119
- This one is probably more definitive and easier to wrap your head around then choosing a specific listing style. First, we must discuss what perspective to write your card and example messages for the bot in: I, You, They. This demonstrates perspective the card is written in - First-person, Second-person, Third-person - and will have noticeable effects on the bot's output. Even cards that are purely list based will still incorporate some form of character perspective, and some are better then others for certain tasks.
120
 
121
- "I" format has the entire card written from the characters perspective, listing things out as if they themselves made it. Useful if you want your bots to act slightly more individualized for one-on-one chats, but requires more thought put into the word choices in order to make sure it is accurate to the way they talk/interact. Most common way people talk online. Keywords: I, my, mine.
122
 
123
- "You" format is telling the bot what they are from your perspective, and is typically the format used in system prompts and technical AI training, but has less outside example data like with "I" in chats/writing, and is less personable as well. Keywords: You, your, you're.
 
124
 
125
- "They" format is the birds-eye view approach commonly found in storytelling. Lots of novel examples in training data. Best for creative writers, and works better in group chats to avoid confusion for the AI on who is/was talking. Keywords: They, their, she/he/its.
126
 
127
- In essence, LLMs are prediction based machines, and the way words are chosen or structured will determine the next probable outcome. Do you want a personable one-on-one chat with your bots? Try "I" as your template. Want a creative writer that will keep track of multiple characters? Use "They" as your format. Want the worst of both worlds, but might be better at technical LLM jobs? Choose "You" format.
128
 
129
- This reasoning also carries over to the chats themselves and how you interact with the bots, though you'd have to use a mixture with "You" format specifically, and that's another reason it might not be as good comparatively speaking, since it will be using two or more styles at once. But there is more to consider still, such as whether to use quotes or asterisks.
130
 
131
- ### 2-4 Should I use quotes or asterisks as the defining separator in the chat?
132
 
133
- Now we must move on to another aspect to consider before creating a character card, and the way you warp the words inside: To use "quotes with speech" and plain text with actions, or plain text with speech and *asterisks with actions*. These two formats are fundamentally opposed with one another, and will draw from separate sources in the LLMs training data, however much that is, due to their predictive nature.
134
 
135
- Quote format is the dominant storytelling format, and will have better prose on average. If your character or archetype originated from literature, or is heavily used in said literature, then wrapping the dialogue in quotes will get you better results.
136
 
137
- Asterisk format is much more niche in comparison, mostly used in RP servers - and not all RP servers will opt for this format either - and brief text chats. If you want your experience to feel more like a texting session, then this one might be for you.
 
 
138
 
139
- Mixing these two - "Like so" *I said* - however, is not advised, as it will eat up extra tokens for no real benefit. No formats that I know of use this in typical training data, and if it does, is extremely rare. Only use if you want to waste tokens/context on word flair.
 
 
140
 
141
- ### 2-5 What combination would you recommend?
142
 
143
- Third-person with quotes for creative writers and group RP chats. First-person with asterisks for simple one-on-one texting chats.
144
 
145
- # 3: Making a Character Card
146
 
147
- ## 3-1 What are some basic sources I should know of before making a character?
 
 
 
 
 
 
148
 
149
- While going in raw when making a character card, either from scratch or from an existing IP, could be fun as an exercise in writing or formatting, its not always practical to do so, and there are a few websites that are easy enough to navigate your way around this to make the process easier. Of course you should probably choose how you would format the card before, like with a listing format in the vein of something like JED+, which was discussed in the last point.
150
 
151
- The first obvious one, if you are using a per-existing character or archetype, is a Wiki or index. Shocking, I know. But its still worth bringing up for beginners. Series or archetypal Wikis can help immensely in gathering info about how your character works in a general sense, and perhaps even bring in new info you wouldn't consider when first starting out. For per-existing characters, just visiting one of the Wikis dedicated to them and dumping it into an assistant to summarize key points could be enough if you just want a base to work with, but you should always check yourself for anything you deem essential for your chat/RP experience in said pages.
152
 
153
- For those that are original in origin, or just too niche for the AI to know what series they hail from, you could always visit separate Wikis or archetypal resources. Is the character inspired by someone else's idea, like some masked vigilante hero who stops crime? Then visiting a "Marvel" or "DC" Wiki or Pedia page that is similar in nature could help with minute details. Say you want to make an elf princess? Maybe the "Zelda" Wiki or Pedia could help. Of course those are more specific cases. There are more general outliers too, like if they are a mermaid or harpy you could try the "Monster Girl Encyclopedia", or if they are an archetype commonly found in TV or Anime you could use "TV Tropes" or "Dere Types Wiki" for ideas. "WebMD" if they have a health or mental condition perhaps, but I'm not a doctor, so ehh...
154
 
155
- I could keep listing sites that might be good for data on archetypes endlessly, but you probably get the picture at this point: If they are based on something else, then there is probably a Wiki or general index to pull ideas from. The next two big ones I'd like to redirect towards are more for helping with specific listings in the appearance and personality sections of you character card.
156
 
157
- ### 3-2 What site should I know about before describing my character's appearance?
158
 
159
- For appearance, visiting an art site like "Danbooru" could help you with picking certain tags for the AI model to read from. Just pick your character, or a character that has a similar build or outfit in mind, and just go from there to help figure out how you want the AI to present your character. Useful if you have a certain outfit or hairstyle in mind, but can't quite figure out what it is called exactly. Not all images will include everything about the clothes or style, so it is important to browse around a bit if you can't find a certain tag you are looking for. While a Wiki might help with this too, Danbooru can get into more specifics that might be lost on the page. There's also that *other* site, which is after 33 and before 35, which has a similar structure if you are really desperate for tags of other things.
160
 
161
- But enough of that for now, how about we move on to the personality section.
 
 
 
 
 
 
 
 
 
 
162
 
163
- ### 3-3 What site should I know about before describing my character's personality?
164
 
165
- For personality, the "Personality Database", while not always accurate, can help give you an idea for how your character might act or present themselves. This is one of those sites I had no idea or cared about beforehand (and still don't to a degree in terms of real life applications) or before LLMs became a thing. Like with Danbooru, even if your character is an OC, just choosing a different character who seems similar to yours might help shape them. Not all of the models used for describing a character's personality will be intrinsically known by an LLM, but there are a few that seem to be universal. However, this might require a bit more insight later on how to piece it all together.
166
 
167
- The big ones used there that most LLMs will be able to figure out if asked are: Four Letter, or "MBTI" as its typically called, which is a a row of letters to denote stuff like extroversion vs introversion, intuition vs sensing, a thinker vs a feeler, and perceptive vs judging. Enneagram, which denotes a numbered type between 1 and 9, along with a secondary wing that acts as an extension of sorts. Temperament is 4 core traits that can be either solitary or combined with a secondary, like with the number typing. Alignment, which is a DnD classification if someone is Lawful or Chaotic, Good or Evil, or something in between with Neutral. And Zodiac, which is probably the most well known, and is usually in coloration with a character's birthday, although that isn't always the case. The others listed on that site are usually too niche, or require extra prompting to get right like with Instinctual Variant.
168
 
169
- If you don't want to delve into these ideas as a standalone yourself, then just dropping those into an assistant bot like before and ask for a summery or keywords relating to the personality provided will help if you need to get your character to tick in a certain way.
 
 
 
 
 
 
 
 
170
 
171
- There are some other factors you could consider as well, like Archetypes specifically again (tsundere, mad genius, spoiled princess, etc. or Jung specifics) and Tarot cards (there are so many articles online when it comes to tarot and zodiac readings that was probably fed into AI models) which are worth considering when asking an AI for a rundown on traits to add.
172
 
173
- You could also combine both the compact personality before you asked the AI assistant, and the complex list it will spit out if you want to double up on traits and not be redundant in wording, which can help with the character's stability. We can probably move on to general findings now.
174
 
175
- ### 3-4 What general ideas are worth considering for my character card?
176
 
177
- We can probably discuss some sub-sections which might be good to list out as a start.
 
 
 
178
 
179
- "Backstory or Background" is one of the more pivotal, but also easy to grasp, section of the card. This helps give the bot a timeline to know how the character evolved before interacting with them, but also at what point of the story they are from if they come from an existing IP.
180
 
181
- "Likes/Dislikes" are another easy one to understand. These will make it so your character will react in certain ways when confronted with them. Individually for both sections works, but you can also make subsections of these as well if they have multiple, like Food, Items, Games, Activities, Actions, Colors, Animals, and Traits, just to name a few. Another way to approach this is have tiers instead, for example a character could have this -Likes Highly: Pizza, Sausage, Mushrooms- But also -Likes Slightly: Pineapple- to denote some semblance of nuance with how they react and choose things.
182
 
183
- "Goals/Fears" are a strong factor which can drive a character in certain ways, or avoid, or even maybe tackle as challenge to overcome later. Main and secondary goals/fears can also, again, help with some nuance.
184
 
185
- "Quirks" are of course cool if you want to differentiate certain actions and situations.
186
 
187
- "Skills/Stats" will help denote what a character is or isn't good at, although stats specifically should maybe be used in a more Adventure/RPG like scenario, though it can still be understood in a mundane sense too.
188
 
189
- "Views" is similar to the personality section, but helps in different and more specific ways. This can be either their general view on things, how they perceive others characters or the user and their relationship with them, or more divisive stances like politics and religion.
 
 
 
190
 
191
- "Speech/Mannerisms" Is probably the last noteworthy one, as this helps separate it from general quirks by themselves, and how they interact with others specifically, which can be used in conjunction with example messages inside the card.
 
192
 
193
- ### 3-5 Are example messages worth adding to a character card?
194
 
195
- If you want your character to stick to a specific way of interacting with others, and help differentiate better in group chats for the AI, then I'd say yes. You could probably get away with just the starting message and those listings above if you want a simple chat, but I've found example messages, if detailed and tailored in the way you prefer for the chat/RP/writing session, will help immensely with getting certain results. Its one thing to list something for the bot to get a grasp of its persona, but having an actual example with all of the little nuances and formatting choices within said chat, will net you better results on average. Prose choice is one big factor in helping the bot along, like the flick of a tail, or the mechanical whirl of a piston arm, can help shape more fantastical characters of course, but subtle things for more grounded characters is of course good too.
196
 
197
- Me personally, I like to have multiple example messages, say in the 3~7 range, and this is for two reasons. One is so the character can express multiple emotions and scenarios that would be relevant to them, and just having to cram it all inside one message might make it come across as schizo in structure, or become a big wall of text that could lead to bloat. And the second is varying message length itself, in order to ensure the bot doesn't get comfortable in a certain range when interacting.
 
 
198
 
199
- # 4 Messages/Conversations With a Bot
 
200
 
201
- ## 4-1 What are some things I should know before chatting with my bots?
202
 
203
- There are quite a few things to discuss, but perhaps one trick we should discuss is something that should happen before you go into any sort of creative endeavor with your bots, and that is doing some Q&A testing with your model of choice. Notice that I said "model" specifically, and not bot/character? Well that's because not all LLMs will have the same amount of data on certain subjects, even if they are the same size or brand. This is probably obvious to most people who've used more than one model/service, but it's important to consider still for newcomers.
 
 
204
 
205
- The basic idea of this activity is to use a blank slate card, typically with something simple like "you/me" as the names of the user/assistant with no other details added, and find out how accurate the depths of its knowledge pool is in certain field that you think are important for your specific case.
206
 
207
- While dry in actual practice, if you want to be the most accurate with your cases, then you should have your settings/samplers turned off or at extremely low to ensure the model doesn't hallucinate too much about any given scenario. If using any settings besides 0, then you should probably swipe a few times to see if the info remains consistent. This goes for both asking the bot about its information, and testing creative models as well, since you might get lucky the first time around.
208
 
209
- As an aside from the last point, and to go on a slight tangent (you can skip to the next section) I've found some people can be misleading when it comes to marketing their own material. Saying the model can do X scenario, but is inconsistent in actual practice. Benchmaxing leaderboards is one field some users have had an issue with, but this extends outside that scope as well, such as saying their model captures the character or writes the scene out very well, but instead personally finding out later that these are most likely cherry picked examples through the use of many swipes. And my preference in determining a model's quality is both creativity AND consistency. It's a shame that a scientific field like LLMs have been infested with grifters wanting to make a name for themselves to farm upvotes/likes, uninformed prompters willfully spreading misinformation because of their own ego, or just those trying to get easy Ko-Fi donations through their unskilled work. But it is what it is I suppose... Now, enough of my personal displeasures - let us get back on track with things to consider before you engage with your model.
 
 
210
 
211
- ### 4-2 What should I ask my bot specifically when it comes to its knowledge?
212
 
213
- To start, world and character knowledge of existing IPs and archetypes, or history and mythology, are big ones for anyone with creative aspirations. As an example, your model probably knows some info about The Legend of Zelda series and fantasy tropes in general, but maybe it doesn't quite get the finer details of the situation or task you are asking about: Wrong clothes or colors, incorrect methodology or actions, weird hallucinations in general, etc.
214
 
215
- The main reason you'd want to know this is to try and save context space with your character cards or world info. If they already know how to play out your character or scene intrinsically, then that's one potential area you can most likely leave out and skip when writing stuff down. This goes for archetypes as well, such as weird creatures or robots, landmarks, history, culture, or personalities that you want to inject into your story.
216
 
217
- You can either ask the bot directly what X thing is, or instead ask it to write a brief scenario/script where the things you are asking about in the first place are utilized within a narrative snippet. This will help give you a better idea on what areas the model excels at, and what it doesn't. You could even ask the bot to make a template of your character or archetype to see what it gets right or wrong. Though you should be on the look out for how it formats things as well.
 
 
 
 
 
 
 
 
218
 
219
- ### 4-3 What should I be on the look out for when a bot formats stuff?
220
 
221
- If you decide to engage with a blank bot, then here is an area if you want to incrementally squeeze out better results from a model: How it formats the story in question and the preferences inside. Does it use quotes or asterisks more often? Does it use regular dashes or em dashes? How does it highlight things if asking for a profile list for your character? Taking into consideration the natural flow of how the model writes things down will inform you better on how it operates, and lets you work better with it, instead of against. Of course this should mostly be considered if you are sticking to a specific model or brand, but there are some the are similar enough in nature to where you won't have to worry about switching.
222
 
223
- ### 4-4 Is there formatting inside the actual chat/rp that I should take into consideration?
224
 
225
- Yes, and these will be more impactful when actually conversing with your bots. Now, formatting isn't just about how it initially starts out with blank bots, but also how the chat develops with actual characters/scenarios. The big one I've noticed is message length. If you notice your bot going on longer then it should, or not long enough, then its possible that the previous messages have made your model get into a groove that will be hard for it to naturally break out of. This is why in the beginning you should have some variance in both the bot's messages and yourself. Even if you are a basic chatter or storyteller, you should still incorporate special symbols beyond basic word characters and the comma/period.
226
 
227
- You should also be mindful of how many times it uses commas as well, since if it only uses one in each sentence it can then get into a groove where it will only use one comma going forward. Once you notice it not being able to use more than one comma in any given sentence, you will never not see it: "I said hello to them, waving as I did. We walked for awhile in the park, looking at the scene around us. It was a pleasant experience, one that was tranquil in nature." This is an example of how the structure has become solidified for the model. Some models are better then others at breaking out, but you should still avoid this if possible. Editing their responses to be more varied, or swiping until the format is different, are some ways to rectify this, but you should also be mindful of your own messages to make sure you aren't doing the same mistakes. Sometimes having Author's Notes will help, but it's still a crap shoot.
 
 
 
 
228
 
229
- ### 4-5 Can I do anything useful with Author's Notes?
230
 
231
- The Author's Note, if your api has one, is one of the more effective ways of getting around bad practices besides the system prompt if tuned to the recent message. If it doesn't, then using a special example container like OOC might work too. Anyway, giving it advice for message length, or guiding it down a certain path is obviously helpful to steer the conversation, but it also helps as a reminder of sorts once the chat gets longer.
232
 
233
- Since it's at the front and easier to access then the initial system prompt, you can think of Author's Notes as miniature version of the system prompt for instructions that are more malleable in nature. You can give it choices to consider going forward, shift the tone with genre tags, remind them of past events, or novel mechanics that are more game centric like current quests or inventory.
 
7
  ---
8
 
9
 
10
+ Greetings! This page intends to share some insights I've gathered from several years of experience testing Large Language Models (LLMs). The focus is primarily on optimizing LLMs for creative writing and roleplaying applications, though these tips may also be beneficial for technical tasks as well.
11
 
12
+ These suggestions may not apply universally to every model or use case, nor do they guarantee optimal results in every instance. However, they should help improve your results and overall experience with the model, even if marginally, and assist you in navigating common pitfalls or misleading advice. Some points may be familiar to experienced users, but I aim to include less commonly discussed details too. Please note that I am still learning and open to feedback and corrections.
13
 
14
+ # 1: Understanding Instruct Formatting
15
 
16
+ ## 1-1: What is an Instruct Format?
17
 
18
+ The Instruct Format (also known as an Instruct Template) is arguably one of the most crucial aspects for ensuring an LLM functions correctly. It defines the specific structure and special tokens used to separate different parts of the input – such as system prompts, user messages, and AI responses – during the model's training and subsequent interactions.
19
 
20
+ Some formats, like ChatML or Alpaca, are widely adopted across various models. Others are specific to certain model families, such as Llama 3 Instruct or Mistral Instruct. However, it's important to note that not all models released under a specific brand necessarily use that brand's official format; fine-tuned versions might employ a different structure.
21
 
22
+ It's crucial to identify the correct format a model expects before using it. You can usually find this information in the model's documentation (e.g., the model card on platforms like Hugging Face). If the format isn't explicitly stated, you can often deduce it by examining the model's configuration files, specifically `tokenizer_config.json` and sometimes `special_tokens_map.json`, located within the model's directory.
23
 
24
+ For instance, finding tokens like `<|im_start|>` and `<|im_end|>` in these configuration files for a model based on Mistral architecture strongly suggests it was fine-tuned using the ChatML format, even if the base Mistral model uses a different native format. Familiarity with common format tokens helps in identifying the correct structure, especially when documentation is missing or unclear.
25
 
26
+ ## 1-2: Should You Deviate from the Prescribed Format?
27
 
28
+ Adhering to the intended Instruct Format generally yields the best performance in terms of coherence, instruction following, and prose quality. However, some users experiment with deviating from the standard format, sometimes claiming it can reduce censorship or elicit different response styles.
29
 
30
+ Often, this deviation comes at the cost of reduced performance and reliability, making it a questionable trade-off. There are typically more effective methods to achieve desired outcomes, such as careful prompt engineering or strategically editing and continuing the AI's responses.
31
 
32
+ Based on my observations, fine-tuning an *existing Instruct-tuned model* (rather than a base model) using a *different* format than the one it was originally trained on (e.g., applying ChatML formatting during fine-tuning on a model originally released with Mistral Instruct) can lead to performance degradation. This results in less optimal responses compared to using a consistent format throughout the fine-tuning process.
33
 
34
+ This potential issue is separate from other challenges that can arise when fine-tuning already instruction-tuned models. However, if the fine-tuning *does* use the underlying model's original format, or if advanced alignment techniques like DPO (Direct Preference Optimization) or its variants (ORPO, etc.) are employed, the resulting model tends to perform more reliably.
35
 
36
+ ## 1-3: Handling Models Listing Multiple Formats
37
+
38
+ Models might list multiple compatible formats due to techniques like model merging or specific fine-tuning choices where the developer intentionally trained across different structures. This is sometimes an intentional training strategy.
39
+
40
+ In such cases, you may need to experiment to determine which format yields the best results for your specific use case. While merging models or formats can sometimes produce unique capabilities, success often depends on careful prompting that aligns with the model's mixed training history.
41
+
42
+ Let me illustrate with a couple of examples:
43
+
44
+ **Example 1: Nous-Capybara-limarpv3-34B and Hybrid Formatting**
45
+
46
+ An older model, Nous-Capybara-limarpv3-34B, exemplifies this situation. It was based on a Vicuna-formatted model but had a LoRA (limarpv3) applied, which introduced a "Message Length Modifier" feature trained using the Alpaca format. This modifier allows users to suggest response length (e.g., `short`, `medium`, `long`) within the Assistant's prompt prefix.
47
+
48
+ The base Capybara model uses the Vicuna format:
49
 
50
  ```
51
  System:
 
53
  Assistant:
54
  ```
55
 
56
+ The limarpv3 LoRA, however, used the Alpaca format for its training data, including the length modifier:
57
 
58
  ```
59
  ### Instruction:
60
  ### Input:
61
+ ### Response: (length = short/medium/long/etc)
62
  ```
63
 
64
+ *(Note: This specific syntax within the response prefix is unique to this model's LoRA).*
65
+
66
+ While the base formats differ, experimentation revealed a way to combine elements. Using the standard Vicuna format while simply adding the length modifier tag did not reliably activate the feature:
67
 
68
  ```
69
  System:
70
  User:
71
+ Assistant: (length = short/medium/long/etc)
72
  ```
73
 
74
+ *(This approach was generally ineffective).*
75
+
76
+ However, by adopting the triple-hash style from Alpaca *within* the Vicuna structure, the length modifier became effective:
77
 
78
  ```
79
  ### System:
 
81
  ### Assistant: (length = short/medium/long/etc)
82
  ```
83
 
84
+ *(This hybrid approach successfully influenced response length).*
85
+
86
+ This demonstrates how elements from different formats used during training or merging might sometimes be combined effectively through careful experimentation.
87
 
88
+ **Example 2: RP-Stew-v4 and Stop Tokens**
89
+
90
+ Another case involves RP-Stew-v4, a model resulting from merging Vicuna and ChatML-based models. The ChatML format looks like this:
91
 
92
  ```
93
  <|im_start|>system
 
98
  Bot response<|im_end|>
99
  ```
100
 
101
+ Note that ChatML explicitly includes roles (`system`, `user`, `assistant`) within its tags and uses `<|im_end|>` as an end-of-turn token.
102
+
103
+ Standard Vicuna format doesn't use explicit end-of-turn tokens. However, for RP-Stew-v4, adding a similar token (`<|end|>`) after each role's content in a Vicuna-like structure proved beneficial:
104
 
105
  ```
106
  SYSTEM: system prompt<|end|>
 
108
  ASSISTANT: assistant output<|end|>
109
  ```
110
 
111
+ This hybrid structure helped reduce rambling, repetitive outputs, and instances of the model incorrectly continuing the user's turn (speaking *as* the user).
112
+
113
+ Generally, sticking to a single, consistent format during model merging yields more predictable results, but these examples show that careful, informed mixing can sometimes be advantageous.
114
+
115
+ ## 1-4: The Importance of Stop Tokens
116
+
117
+ In my experience, models often perform more reliably when their format includes explicit stop tokens (or end-of-turn tokens). As demonstrated with the RP-Stew-v4 example, adding appropriate stop tokens significantly reduced unwanted repetition (an estimated 25-33% reduction in rambling length was observed in that specific case).
118
+
119
+ Formats incorporating stop tokens generally provide clearer structural boundaries for the model. This helps it recognize the end of a conversational turn and maintain role distinction more consistently, leading to greater stability, especially in creative back-and-forth exchanges.
120
+
121
+ Conversely, omitting necessary stop tokens can sometimes lead to run-on responses, confusion between roles, or other unpredictable behavior. While experimentation is always possible, deviating significantly from the intended token structure often negatively impacts overall performance.
122
+
123
+ When experimenting with formats and tokens, it's wise to use separate configurations or instances to avoid errors. Incorrect placement (e.g., putting a stop token *before* the user's input within the user prefix) can lead to unexpected and incorrect model behavior.
124
+
125
+ # 2: Character and Chat Formatting
126
+
127
+ ## 2-1: What is a Character Card?
128
+
129
+ Although somewhat obvious, let's define Character Cards first. Character Cards establish the persona for an AI bot to impersonate. This persona can represent a real person, a character from an established franchise, or an original character (OC).
130
+
131
+ Character Cards are typically structured like a profile or dossier, outlining the character's key attributes. Various formatting styles can be used to detail their unique traits, background, and mannerisms.
132
+
133
+ ## 2-2: What are the Different Styles of Character Cards?
134
+
135
+ Creating effective Character Cards is not an exact science; the best approach can vary depending on the specific LLM (brand, size) and the desired outcome. However, several popular styles have emerged within the user community.
136
+
137
+ * **Prose Style:** One style involves writing the character description in natural prose, much like a narrative description in a book. This method requires careful writing to ensure clarity, avoid excessive keyword repetition, and maintain flow. It might be more challenging for beginners but potentially well-suited for users focused purely on narrative writing.
138
 
139
+ * **List Formats:** Another common approach uses list formats, where traits are categorized clearly. Variations exist, including:
140
+ * Simple Markdown lists (`- item` or `* item`).
141
+ * Wiki-style formatting (using templates or specific syntax like `== Section ==`).
142
+ * Community-created structured formats (like JED+ or W++).
143
 
144
+ These list styles might use parentheses `()` or brackets `[]` to enclose information, dashes `-` or asterisks `*` for list items, or formatting like bolding (`**text**` or `### Heading`) to structure sections.
145
 
146
+ Determining the 'best' format is difficult, as effectiveness can be model-dependent. However, based on my experience, highly structured but non-standard formats like W++ can sometimes lead to less consistent or coherent behavior (perhaps due to being less common in the LLM's training data). Similarly, directly pasting large amounts of raw text from a wiki ("wiki style") without curation can introduce irrelevant information ('bloat') and may not be optimally processed by the model.
147
 
148
+ My current recommendation is something akin to JED+, which often involves a **hybrid approach**: using lists for concrete details like appearance and personality traits, and employing standard prose for background history or defining speech patterns.
149
 
150
+ Regardless of style, it's essential to consider the writing perspective used within the card.
151
 
152
+ ## 2-3: What Writing Perspectives Should I Consider?
153
 
154
+ Choosing the writing perspective for your Character Card is arguably more straightforward than selecting a specific formatting style, but it's equally crucial. The main options are:
155
 
156
+ * **First-Person ("I"):** The card is written from the character's own viewpoint (e.g., "I am...", "My background is...").
157
+ * *Pros:* Can lead to bots with a more distinct, individualized voice, suitable for one-on-one chats. Reflects common online communication styles.
158
+ * *Cons:* Requires careful crafting to accurately reflect the character's unique voice and mannerisms.
159
+ * *Keywords:* `I`, `me`, `my`, `mine`.
160
 
161
+ * **Second-Person ("You"):** The card addresses the bot directly (e.g., "You are...", "Your personality includes...").
162
+ * *Pros:* Often used in system prompts and AI training for direct instruction. Can be effective for defining behavior explicitly.
163
+ * *Cons:* Less common in creative writing or typical chat data, potentially feeling less personal or natural for roleplaying. Can lead to awkward perspective mixing during interaction.
164
+ * *Keywords:* `You`, `your`, `yours`, `you're`.
165
 
166
+ * **Third-Person ("They"/"He"/"She"):** The card describes the character objectively (e.g., "He is...", "Her history involves...").
167
+ * *Pros:* Standard perspective for storytelling and literature, abundant in training data. Excellent for creative writing and often better for managing multiple characters in group chats due to clear distinction.
168
+ * *Cons:* May feel slightly less direct for simple one-on-one interaction compared to First-Person.
169
+ * *Keywords:* `He/She/They`, `her/his/their`, `hers/his/theirs`, `it/its`.
170
 
171
+ Even list-based cards implicitly adopt a perspective through pronoun usage and phrasing within the descriptions. Because LLMs predict text based on patterns, the chosen perspective significantly influences the likely style of the bot's responses.
172
 
173
+ * For **personal one-on-one chats**, First-person ("I") often works well.
174
+ * For **narrative writing or multi-character scenarios**, Third-person ("They") is generally preferred due to clarity and alignment with literary training data.
175
+ * **Second-person ("You")** is less common for character interaction and arguably the worse overall, but can be useful for direct instruction.
176
 
177
+ This principle extends to your own messages during the chat. Interacting consistently with the chosen perspective generally works best. Using a Second-person ("You") card often necessitates a mix of perspectives in the interaction (e.g., addressing the bot as "You" while narrating your actions in third person), which can sometimes confuse the model.
178
 
179
+ Another crucial formatting choice involves how dialogue and actions are represented in the chat.
180
 
181
+ ## 2-4: Should I Use Quotes or Asterisks for Dialogue/Actions in Chat?
182
 
183
+ Next, consider how to format dialogue and actions within the chat interaction itself. The two primary styles are:
184
 
185
+ 1. **Quote Style:** `"Dialogue is enclosed in quotes,"` `while actions are described in plain text outside the quotes.`
186
+ 2. **Asterisk Style:** `Dialogue is written in plain text,` `*while actions are enclosed in asterisks.*`
187
 
188
+ These styles are fundamentally distinct and likely draw upon different subsets of the LLM's training data due to the strong patterns they represent.
189
 
190
+ * **Quote Style:** This is the standard format in literature and published fiction. Consequently, using it often results in better prose quality and narrative coherence, especially if the character or archetype is common in literary works. Using quotes for dialogue is generally recommended for story-writing or more formal roleplaying.
191
 
192
+ * **Asterisk Style:** This style is comparatively niche, frequently seen in online roleplaying communities (though not universally) and resembling instant messaging or script-like conventions. It can be suitable if you prefer a style closer to text-based RP or casual chat.
193
 
194
+ **Mixing the styles** within a single message (e.g., `"Quoted dialogue" *action in asterisks*`) is **strongly discouraged**. This hybrid approach is generally absent from standard training datasets, offers no significant benefit, consumes extra tokens, and potentially confuses the model. Avoid using it unless you want to emphasize certain sections of the dialogue or text to make them distinct.
195
 
196
+ ## 2-5: Recommended Combinations
197
 
198
+ Based on the points above, here are my general recommendations for combining perspective and chat style:
199
 
200
+ * **For Creative Writing / Group Roleplay:**
201
+ * Use **Third-person** (`They/She/He`) perspective in the Character Card.
202
+ * Use the **Quote Style** (`"Dialogue"`, `Action`) for chat interactions.
203
 
204
+ * **For Simpler One-on-One / Texting-Style Roleplay:**
205
+ * Use **First-person** (`I`) perspective in the Character Card.
206
+ * Use the **Asterisk Style** (`Dialogue`, `*Action*`) for chat interactions.
207
 
208
+ # 3: Crafting a Character Card
209
 
210
+ ## 3-1: Essential Resources for Character Creation
211
 
212
+ While creating a Character Card entirely from scratch, whether for an original character (OC) or one from an existing intellectual property (IP), can be a valuable writing exercise, it's not always the most practical approach. Several online resources can help streamline the process. Of course, it's best to decide on your preferred Character Card format (as discussed in Part 2) before gathering information.
213
 
214
+ * **Wikis and Indexes:** For existing characters or established archetypes, wikis are an obvious but crucial starting point, especially for beginners. Fan wikis (like those on Fandom) or encyclopedic sites (like Wikipedia) provide comprehensive information, helping you grasp the character's overall concept and potentially uncovering details you hadn't considered.
215
+ * For **existing characters**, you can visit a wiki dedicated to their source material. Using an AI assistant to summarize key points from a relevant wiki page can provide a good base, but always review the summary and the original pages yourself to ensure essential details for your intended chat/RP experience are included.
216
+ * For **OCs or niche characters** without dedicated wikis, consult broader resources. Is your character inspired by a known archetype (e.g., a masked vigilante)? Visiting wikis for similar established characters (like from Marvel or DC Fandom) can offer inspiration for finer details. Need ideas for an elf princess? Perhaps the Zelda Wiki could offer relevant concepts.
217
+ * **Archetype/Trope Resources:** More general resources can also be invaluable:
218
+ * For specific mythical or fantasy beings (e.g., mermaids, harpies), specialized encyclopedias might exist (like the "Monster Girl Encyclopedia" wiki, focusing on specific interpretations).
219
+ * For common character types found in media, sites like "TV Tropes" (for narrative patterns and character archetypes) or the "Dere Types Wiki" (for specific anime/manga personality archetypes) can provide ideas and terminology.
220
+ * For characters with specific health or psychological conditions, consulting relevant, specialized resources could provide simple examples if wanted, like WebMD for defining complex traits.
221
 
222
+ The underlying principle is: if your character is based on an existing concept or archetype, there's likely a wiki or index online you can draw inspiration from. Beyond these general information sources, let's highlight two types of sites useful for specific card sections.
223
 
224
+ ## 3-2: Resources for Describing Appearance
225
 
226
+ To detail your character's physical appearance, art databases with tagging systems can be very helpful.
227
 
228
+ * **Image Tag Sites (e.g., Danbooru):** Sites like Danbooru allow you to search for characters (yours, if they have art, or similar ones) and identify specific tags related to their build, clothing, hairstyle, accessories, etc. This is useful if you have a visual in mind but lack the precise terminology. Not every image is exhaustively tagged, so browsing multiple images might be necessary. While a wiki might describe appearance generally, tag sites can offer more specific visual keywords that LLMs might recognize (e.g., "blue sailor collar," "thighhighs," "long low-tied twintails").
229
 
230
+ ## 3-3: Resources for Describing Personality
231
 
232
+ Defining personality can be complex, but certain resources offer frameworks and keywords.
233
 
234
+ * **Personality Database (PDB):** This website catalogs characters (real and fictional) according to various personality typology systems. While user-submitted typings aren't always perfectly accurate and the real-world validity of some systems is debated, PDB can be a useful tool for LLM character creation. Even if your character is an OC, browsing profiles of characters with similar perceived traits can provide inspiration and terminology.
235
+ * **Common Typology Systems:** Several systems listed on PDB are often recognizable by LLMs:
236
+ * **MBTI (Myers-Briggs Type Indicator):** Uses a four-letter code (e.g., INTP, ESFJ) representing preferences across four dichotomies (Introversion vs. Extraversion, Intuition vs. Sensing, Thinking vs. Feeling, Perceiving vs. Judging).
237
+ * **Enneagram:** Describes nine core personality types (Type 1 to 9), often supplemented by a "wing" (one of the adjacent types, e.g., 5w4, 9w1) indicating nuances.
238
+ * **Temperaments:** Often refers to the Four Temperaments (Sanguine, Choleric, Melancholic, Phlegmatic), sometimes used in combination like the wing system.
239
+ * **Alignment:** Originating from Dungeons & Dragons, classifies characters on two axes: Lawful vs. Chaotic and Good vs. Evil (resulting in types like Lawful Good, Chaotic Neutral, Neutral Evil).
240
+ * **Zodiac:** The familiar astrological signs, often associated with a character's birthday, but sometimes used purely thematically.
241
+ * **Other Personality Concepts:**
242
+ * **Instinctual Variants (Enneagram Subtypes):** May be too niche or require specific prompting.
243
+ * **Archetypes:** Specific character roles or patterns (e.g., Tsundere, Mad Genius, Spoiled Princess; or Jungian archetypes like The Hero, The Mentor).
244
+ * **Tarot Associations:** The symbolic meanings of Tarot cards are widely discussed online and likely present in training data.
245
 
246
+ **Using These Resources:** You don't necessarily need to become an expert in these systems. You can often use an AI assistant effectively by providing it with a character's profile from PDB (e.g., "INTP 5w4 Lawful Neutral") and asking it to summarize the associated traits or generate keywords. You can then incorporate these keywords or summary points into your Character Card's personality section. Combining the "compact" typology codes with the "complex" list of generated traits can help reinforce the desired personality without excessive token repetition.
247
 
248
+ ## 3-4: Key Sections for Your Character Card
249
 
250
+ Consider including some of the following sections in your card for a well-rounded character definition:
251
 
252
+ * **Backstory / Background:** A crucial yet straightforward section. It provides context on the character's history, development, and potentially their current situation or point in their original timeline (if from an existing IP).
253
+ * **Personality:** Describes core traits, temperament, and demeanor (potentially drawing from the typology systems mentioned above).
254
+ * **Appearance:** Details physical features, clothing, and overall visual style (potentially using keywords from tag sites).
255
+ * **Likes / Dislikes:** Guides how the character reacts to specific things. Listing items separately works well. You can also use subcategories (e.g., Food, Activities, People, Concepts) or tiers (e.g., `Loves: [item]`, `Likes: [item]`, `Tolerates: [item]`, `Dislikes: [item]`, `Hates: [item]`) to add nuance.
256
+ * **Goals / Fears:** Strong drivers for character motivation and behavior. Defining primary and secondary goals/fears can add depth. These can represent aspirations to achieve or challenges to avoid or confront.
257
+ * **Quirks / Habits:** Unique mannerisms, routines, or distinctive behaviors that differentiate the character.
258
+ * **Skills / Abilities / Stats:** Defines what the character is capable of. Skills/Abilities apply broadly, while numerical Stats are often more suited for RPG or adventure settings but can sometimes be adapted for everyday contexts (e.g., `Stamina: Low`).
259
+ * **Views / Beliefs:** Covers the character's opinions on various topics, relationships with specific other characters (including the user), or stances on more significant themes like politics, religion, or morality. Helps refine personality in specific contexts.
260
+ * **Speech / Mannerisms:** Focuses specifically on communicative habits – tone of voice, common phrases, accent, verbal tics, etc. This works in conjunction with example messages.
261
 
262
+ ## 3-5: The Value of Example Messages
263
 
264
+ Are example messages essential? While you *might* suffice with just the core description and a starting message for simple chats, **adding detailed example messages is highly recommended.**
265
 
266
+ If you want the bot to consistently adhere to a specific interaction style, voice, and formatting, examples are invaluable. Listing traits provides a foundation, but concrete examples demonstrating those traits *in action* – including your chosen chat format (Quotes vs. Asterisks), prose style, and typical response length – yield significantly more reliable results. They help the AI grasp the nuances of the character's expression.
267
 
268
+ * **Prose Detail:** Examples allow you to showcase descriptive language fitting the character (e.g., "her tail gave a thoughtful flick," "the piston in his arm whirred softly") or more subtle cues for grounded characters.
269
+ * **Multiple Examples:** Providing several examples (e.g., 3-7 messages) is often beneficial.
270
+ 1. It allows you to demonstrate the character reacting to different situations or expressing various relevant emotions without cramming everything into one potentially disjointed or overly long message.
271
+ 2. It allows you to vary example message length, which can encourage the bot to generate responses of varying lengths itself, rather than settling into a repetitive pattern.
272
 
273
+ Example messages bridge the gap between description and performance, significantly improving the consistency and believability of the AI's portrayal.
274
 
275
+ # 4: Interacting with the Bot: Messages and Conversations
276
 
277
+ ## 4-1: Pre-Chat Preparation: Testing the Model's Knowledge
278
 
279
+ Before diving into creative interactions with your character bots, consider performing some preliminary tests on the underlying LLM itself. Why test the *model* rather than the specific *character*? Because different LLMs, even of the same size or family, possess varying levels of knowledge on specific subjects based on their training data. Understanding the model's baseline knowledge is crucial, especially for newcomers.
280
 
281
+ **The Testing Process:**
282
 
283
+ 1. **Use a Blank Slate:** Set up a minimal chat configuration. Use simple names like "User" and "Assistant" (or "You" and "Me") for the participants and provide *no* character description or system prompt details beyond the basic Instruct Format structure.
284
+ 2. **Query Key Topics:** Ask the model direct questions about subjects relevant to your intended creative work (e.g., specific lore from an IP, historical events, scientific concepts, character archetypes).
285
+ 3. **Minimize Randomness:** To assess factual recall accurately, set the model's sampling temperature to 0 or extremely low (e.g., 0.01). This reduces the chance of the model hallucinating or inventing information.
286
+ 4. **Check Consistency:** Even with low temperature, it's wise to generate ("swipe") a few responses to the same factual query to see if the information remains consistent. If testing creative generation (as discussed below), consistency across multiple swipes is also important.
287
 
288
+ **A Note on Model Consistency and Marketing:**
289
+ (You can skip to the next section if preferred.) Related to testing, it's worth noting that model performance can vary. Sometimes, advertised capabilities or user-shared examples might not consistently reflect typical performance. Impressive results showcased online could be cherry-picked after many generation attempts ("swipes"). While benchmark scores offer some indication, real-world creative consistency is also vital. My preference is for models that demonstrate both creativity *and* reliability. It's advisable to test models yourself for your specific use cases rather than relying solely on promotional claims or anecdotal reports. Now, returning to the main topic...
290
 
291
+ ## 4-2: What Specific Knowledge Should I Probe?
292
 
293
+ When testing the model's baseline knowledge, focus on areas critical to your creative goals:
294
 
295
+ * **World/Character Knowledge:** How familiar is the model with the specific IPs, characters, locations, or established archetypes you plan to use? Does it understand key historical or mythological contexts?
296
+ * *Example:* Does the model know details about *The Legend of Zelda* series and general fantasy tropes? Test its understanding of specific character appearances, motivations, or key plot points relevant to your scenario. Look for inaccuracies: wrong details, incorrect actions, or nonsensical hallucinations.
297
+ * **Archetypes and Concepts:** Test knowledge of specific creature types (e.g., unique fantasy races, robot designs), cultural practices, historical periods, personality types, or abstract concepts you intend to feature.
298
 
299
+ **Why Test This?**
300
+ The primary goal is **context optimization**. If the model *already* possesses accurate intrinsic knowledge about a character, setting, or concept, you may not need to explicitly detail it in your Character Card or world info prompts. This saves valuable context space and allows you to focus prompts on unique aspects or specific instructions. Conversely, identifying knowledge gaps tells you where you *must* provide explicit information.
301
 
302
+ **How to Test:**
303
 
304
+ * **Direct Questions:** Ask "What is [X]?" or "Describe [character/place/concept]."
305
+ * **Scenario Prompts:** Ask the model to write a short scene or dialogue snippet incorporating the elements you're testing. This reveals not only if it *knows* the information but also if it can *apply* it correctly in a narrative context.
306
+ * **Template Generation:** Ask the model to generate a basic character template or profile for a specific character or archetype. Compare its output against known facts to see what it gets right or wrong. Pay attention to how it formats the information as well.
307
 
308
+ ## 4-3: Observing the Model's Default Formatting
309
 
310
+ When interacting with a blank bot (no character info, minimal system prompt), pay attention to its inherent formatting preferences:
311
 
312
+ * **Dialogue/Action Style:** Does it default to using quotes (`"Dialogue"`) or asterisks (`*Action*`)?
313
+ * **Dashes:** Does it use hyphens (`-`) or em dashes (`—`) for pauses or parentheticals?
314
+ * **List Formatting:** If asked to create a profile or list, what structure does it use (bullet points, bolding, specific syntax)?
315
 
316
+ Understanding a model's natural tendencies can help you work *with* its patterns rather than against them, potentially leading to smoother interactions. For example, if a model consistently uses em dashes, adopting them yourself *might* encourage more consistent output. This is particularly relevant if you plan to primarily use one specific model or family of models with similar training.
317
 
318
+ ## 4-4: Managing Formatting Dynamics Within the Chat
319
 
320
+ Formatting consistency is crucial not just at the start but throughout an ongoing conversation or roleplay. Be mindful of these common issues:
321
 
322
+ * **Message Length Inertia:** Models can sometimes fall into a pattern based on recent message lengths. If your recent messages and the bot's replies have all been short, it might struggle to generate longer, more detailed responses when needed (and vice-versa).
323
+ * *Solution:* Intentionally vary the length and complexity of your own messages from the beginning of the chat. Don't let every turn be roughly the same length.
324
+ * **Repetitive Sentence Structure (e.g., "The Comma Groove"):** Models can sometimes adopt overly simplistic or repetitive sentence structures, often signaled by using exactly one comma per sentence, regardless of complexity.
325
+ * *Example:* "I greeted him, smiling warmly. We walked through the woods, enjoying the quiet path. It was a nice day, perfect for a stroll." Once this pattern sets in, it can be hard for the model to break naturally, and it becomes very noticeable.
326
+ * *Solution:*
327
+ * **Vary Your Input:** Ensure your own messages use diverse sentence structures and punctuation (periods, commas, semicolons, question marks, exclamation points, em dashes).
328
+ * **Edit Bot Responses:** Correct repetitive structures in the bot's replies before continuing.
329
+ * **Swipe/Regenerate:** Generate new responses until you get one with better structure.
330
+ * **Author's Notes (see below):** Sometimes explicit instructions can help.
331
 
332
+ Be mindful that your own input significantly influences the bot's output style over time. Avoid falling into the same repetitive patterns you want the bot to avoid.
333
 
334
+ ## 4-5: Using Author's Notes Effectively
335
 
336
+ Author's Notes (A/N) or similar mechanisms like OOC (Out Of Character) comments, if supported by your interface, can be powerful tools for guiding the conversation dynamically. They are typically inserted near the most recent message and provide context or instructions relevant *to the next turn*, acting as a more flexible, short-term counterpart to the main System Prompt.
337
 
338
+ **Common Uses:**
339
 
340
+ * **Steering the Narrative:** Suggest plot developments, introduce events, or guide the bot towards a specific outcome ("`[A/N: Have the storm begin in the next message.]`").
341
+ * **Adjusting Tone/Style:** Specify a desired mood, genre, or writing style ("`[A/N: Shift tone to be more suspenseful.]`" or "`[OOC: Please use more descriptive prose.]`").
342
+ * **Managing Message Length:** Request shorter or longer responses ("`[A/N: Keep the next reply brief.]`").
343
+ * **Reminders:** Briefly remind the bot of recent events, character states, or established details that might be getting lost further back in the context ("`[A/N: Remember, Character B is injured.]`").
344
+ * **Implementing Mechanics:** For game-centric interactions, use A/N to update status, track inventory, or remind the bot of rules ("`[A/N: User's current quest is 'Find the Lost Artifact'. Inventory contains: Healing Potion x2.]`").
345
 
346
+ Think of Author's Notes as targeted, temporary instructions that help fine-tune the bot's next response without altering the core character definition or overall scenario rules set in the main prompt.
347
 
348
+ That will do it. I hope this information was helpful to you in some way. Link below is if you want info on my personal system prompt and settings.
349
 
350
+ https://huggingface.co/ParasiticRogue/General-model-and-character-settings