--- license: apache-2.0 language: - en tags: - reasoning - thinking - uncensored - gated - mixture of experts - moe - 8x3B - Llama 3.2 MOE - NEO Imatrix - 128k context - creative - creative writing - fiction writing - plot generation - sub-plot generation - fiction writing - story generation - scene continue - storytelling - fiction story - science fiction - romance - all genres - story - writing - vivid prosing - vivid writing - fiction - roleplaying - float32 - swearing - rp - horror - mergekit pipeline_tag: text-generation --- (quants uploading (set#1), model card updates pending, "gating" access/methods, one.) WARNING: NSFW. Vivid prose. INTENSE. Visceral Details. Light HORROR. Swearing. UNCENSORED... humor, romance, fun.

Llama-3.2-8X3B-GATED-MOE-NEO-Reasoning-Dark-Champion-uncensored-18.4B-IMAT-GGUF

IMPORTANT: This model has on/off/variable control reasoning from NousResearch and the DeepHermes model, and requires a system prompt(s) as provided to invoke reasoning/thinking. Please see operating instructions below for best performance. It is a LLama 3.2 model, max context of 128k (131,000) using mixture of experts to combine EIGHT top L3.2 3B models into one massive powerhouse at 18.4B parameters (equal to 24B - 8 X 3B). This model's instruction following, and output generation for creative writing, prose, fiction and role play are exceptional. This model is also "gated", contains a master reasoning model (this can be turned on/off), was built at float32 (32 bit) precision and quants have the output tensor at Q8_0, with a few choice quants at f16 (16 bit) and a Q8_0 with f32 (32 bit). These quants are also mastered using the NEO Imatrix dataset, including new methods to "imatrix" both the output tensor and tokens too. The "gated" strucure means the "reasoning model" is re-inforced by the other 7 models in the MOE during reasoning, and then during output generation / non-reasoning the non-reasoning model(s) take control. Also, with "gating" you can directly access/control the model(s) you want to use during instruction following and generation. This model is the "reasoning / gated version" of this model: [ https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF ] (193 likes // 447,000+ downloads to date. (Hugging face only shows last 30 days at the repo) ) And it is fast: 50+ t/s (2 experts) on a low end 16GB card, IQ4XS. Double this speed for standard/mid-range video cards. This model can be used also for all genres (examples below showing this). It is for any writing, fiction or roleplay activity. This model can also be used for general use, however its output generation can be uncensored. This model has been designed to be relatively bullet proof and operates with all parameters, including temp settings from 0 to 5. It is an extraordinary compressed model, with a very low perplexity level (lower than Meta Llama3 Instruct). It requires Llama3 template and/or "Command-R" template. Several outputs below, including 2, 4 and 8 experts used. Model Notes: - Detail, prose and fiction writing abilities are OFF THE SCALE relative to all Llama 3.2 models, and many L 3.1, L3 8B+ models. - For more varied prose (sentence/paragraph/dialog) raise the temp and/or add more instructions in your prompt(s). - Role-players: Careful raising temp too high as it may affect instruction following. - This model works with rep pen of 1 or higher, 1.02+ recommended. - If you want a specific type of prose (IE horror) add in "(vivid horror)" or "(graphic vivid horror)" (no quotes) in your prompt(s). - A lot of GPTisms have been removed. There are still a few however - errrrr. Higher "temps" will help with this issue. - This is not a "happy ever after" model but it is also not "horror". It has a light negative bias. - Output length will vary however this model prefers slightly longer outputs unless you state the size. - For creative uses, different quants will produce slightly different output. - Due to the high stability and compressed nature of this model, all quants will operate at above average levels. - Source code for this model and Imatrix GGUFs versions will be uploaded shortly at separate repos. Meet the Team: Mixture of Experts Models This model is comprised of the following 8 models ("the experts") (in full): https://huggingface.co/huihui-ai/Llama-3.2-3B-Instruct-abliterated - https://huggingface.co/NousResearch/DeepHermes-3-Llama-3-3B-Preview [reasoning model] - https://huggingface.co/Hastagaras/L3.2-JametMini-3B-MK.I - https://huggingface.co/ValiantLabs/Llama3.2-3B-Enigma - https://huggingface.co/Hastagaras/L3.2-JametMini-3B-MK.III - https://huggingface.co/huihui-ai/Llama-3.2-3B-Instruct-abliterated - https://huggingface.co/chuanli11/Llama-3.2-3B-Instruct-uncensored - https://huggingface.co/Lyte/Llama-3.2-3B-Overthinker - https://huggingface.co/prithivMLmods/Llama-3.2-3B-Promptist-Mini The mixture of experts is set at 2 experts, but you can use 3,4,5,6.. 7 and even 8. This "team" has a Captain (first listed model), and then all the team members contribute to the to "token" choice billions of times per second. Note the Captain also contributes too. Think of 2, 3 or 4 (or more) master chefs in the kitchen all competing to make the best dish for you. This results in higher quality generation. This also results in many cases in higher quality instruction following too. That means the power of every model is available during instruction and output generation. NOTE: You can use one "expert" too ; however this means the model will randomly select an expert to use EACH TIME, resulting in very different generation for each prompt / regen of a prompt. CHANGING THE NUMBER OF EXPERTS: You can set the number of experts in LMStudio (https://lmstudio.ai) at the "load" screen and via other apps/llm apps by setting "Experts" or "Number of Experts". For Text-Generation-Webui (https://github.com/oobabooga/text-generation-webui) you set the number of experts at the loading screen page. For KolboldCPP (https://github.com/LostRuins/koboldcpp) Version 1.8+ , on the load screen, click on "TOKENS", you can set experts on this page, and the launch the model. For server.exe / Llama-server.exe (Llamacpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md ) add the following to the command line to start the "llamacpp server" (CLI): "--override-kv llama.expert_used_count=int:6" (no quotes, where "6" is the number of experts to use) When using "API", you set the "num_experts_used" in the JSON payload (this maybe different for different back ends). CREDITS: Special thanks to all the model makers / creators listed above. Please visit each repo above to see what model(s) contributed to each of models above and/or to learn more about the models from the model makers. Special credit goes to MERGEKIT, without you this project / model would not have been possible. [ https://github.com/arcee-ai/mergekit ] Special Operations Notes for this MOE model: Because of how this "MOE" model is configured, even though the default is 2 experts, the "selected" 2 will vary during generation. (same applies if you change the number of experts used) This results in vastly different output generation PER generation of each prompt. This is a positive in terms of variety, but also means it may take 2-4 regens (of the same prompt) to get the highest quality. In addition, this model responds very well to Dry, Dynamic Temp, and Smooth/Quadratic samplers. Using these in conjunction with the model can vastly improve output quality. Higher temps (above 1) can also aid in generation - especially word choice/sentence generation. When you increase the number of experts used output quality will also increase, at the cost of tokens per second speed. As you increase/decrease the number of experts, you may want to adjust temp, samplers, and advanced samplers too. Your quant choice(s) too will impact instruction following and output generation roughly this means the model will understand more nuanced instructions and output stronger generation the higher you go up in quant(s). FLASH ATTENTION ENHANCEMENT: As per user feedback here [ https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF/discussions/1 ] I would suggest trying this model with Flash Attention "on", depending on your use case. Quants, Samplers, Generational steering and other topics are covered in the section below: "Highest Quality Settings..." Censored / Uncensored / Abliterated: This model contains several uncensored and/or Abliterated models. As a result is can output uncensored material. However there are a few "censored" models which can sometimes interfer, so here is a how to address this: 1 - Regen your prompt a few times. 2 - INCREASE the number of experts used. What can I use this model for ? This model can be used for fiction writing, any creative prose and role play. It can also be used for just about any general fiction (all genres) activity including: - scene generation - scene continuation - creative writing - fiction writing - plot generation - sub-plot generation - fiction writing - story generation - storytelling - writing - fiction - roleplaying - rp - graphic horror - horror - dark humor - nsfw - and can be used for any genre(s). QUANTS: This repo contains regular quants. For more information on quants, quants choices, and LLM/AI apps to "run" quants see the section below: "Highest Quality Settings..." --- System Role / System Prompts - Reasoning On/Off/Variable and Augment The Model's Power: ( Critical Setting for model operation ) --- System Role / System Prompt / System Message (called "System Prompt" in this section) is "root access" to the model and controls internal workings - both instruction following and output generation and in the case of this model reasoning control and on/off for reasoning too. In this section I will show you basic, advanced, and combined "code" to control the model's reasoning, instruction following and output generation. If you do not set a "system prompt", reasoning/thinking will be OFF by default, and the model will operate like a normal LLM. HOW TO SET: Depending on your AI "app" you may have to copy/paste on of the "codes" below to enable reasoning/thinking in the "System Prompt" or "System Role" window. In Lmstudio set/activate "Power User" or "Developer" mode to access, copy/paste to System Prompt Box. In SillyTavern go to the "template page" ("A") , activate "system prompt" and enter the text in the prompt box. In Ollama see [ https://github.com/ollama/ollama/blob/main/README.md ] ; and setting the "system message". In Koboldcpp, load the model, start it, go to settings -> select "Llama 3 Chat"/"Command-R" and enter the text in the "sys prompt" box. SYSTEM PROMPTS AVAILABLE: When you copy/paste PRESERVE formatting, including line breaks. If you want to edit/adjust these only do so in NOTEPAD OR the LLM App directly. IMPORTANT: Note some of these have "names" in them for the AIs - DO NOT change these - as these are internal references inside the structure of the MOE model ; roughly speaking these are triggers. SIMPLE: This is the generic system prompt used for generation and testing [no reasoning]: 
You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.
This System Role/Prompt will give you "basic thinking/reasoning" [basic reasoning]: 
You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.
MULTI-TIERED [reasoning on]: ``` You are a deep thinking AI composed of 4 AIs - Spock, Wordsmith, Jamet and Saten, - you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself (and 4 partners) via systematic reasoning processes (display all 4 partner thoughts) to help come to a correct solution prior to answering. Select one partner to think deeply about the points brought up by the other 3 partners to plan an in-depth solution. You should enclose your thoughts and internal monologue inside tags, and then provide your solution or response to the problem using your skillsets and critical instructions. ``` MULTI-TIERED - CREATIVE [reasoning on]: ``` Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. As a deep thinking AI composed of 4 AIs - Spock, Wordsmith, Jamet and Saten, - you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself (and 4 partners) via systematic reasoning processes (display all 4 partner thoughts) to help come to a correct solution prior to answering. Select one partner to think deeply about the points brought up by the other 3 partners to plan an in-depth solution. You should enclose your thoughts and internal monologue inside tags, and then provide your solution or response to the problem using your skillsets and critical instructions. Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story. ``` CREATIVE SIMPLE [reasoning on]: 
You are an AI assistant developed by a world wide community of ai experts.

Your primary directive is to provide highly creative, well-reasoned, structured, and extensively detailed responses.

Formatting Requirements:

1. Always structure your replies using: <think>{reasoning}</think>{answer}
2. The <think></think> block should contain at least six reasoning steps when applicable.
3. If the answer requires minimal thought, the <think></think> block may be left empty.
4. The user does not see the <think></think> section. Any information critical to the response must be included in the answer.
5. If you notice that you have engaged in circular reasoning or repetition, immediately terminate {reasoning} with a </think> and proceed to the {answer}

Response Guidelines:

1. Detailed and Structured: Use rich Markdown formatting for clarity and readability.
2. Creative and Logical Approach: Your explanations should reflect the depth and precision of the greatest creative minds first.
3. Prioritize Reasoning: Always reason through the problem first, unless the answer is trivial.
4. Concise yet Complete: Ensure responses are informative, yet to the point without unnecessary elaboration.
5. Maintain a professional, intelligent, and analytical tone in all interactions.
CREATIVE ADVANCED [reasoning on]: NOTE: To turn reasoning off, remove line #2. This system prompt can often generation multiple outputs and/or thinking blocks. ``` Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. You may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside tags, and then provide your solution or response to the problem Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story. ``` --- Template: This is a LLAMA3 model, and requires Llama3 template, but may work with other template(s). If you use "Command-R" template your output will be very different from using "Llama3" template. Here is the standard LLAMA3 template: 
{
  "name": "Llama 3",
  "inference_params": {
    "input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
    "input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
    "pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
    "pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
    "pre_prompt_suffix": "<|eot_id|>",
    "antiprompt": [
      "<|start_header_id|>",
      "<|eot_id|>"
    ]
  }
}
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model: In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ; Set the "Smoothing_factor" to 1.5 : in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F" : in text-generation-webui -> parameters -> lower right. : In Silly Tavern this is called: "Smoothing" NOTE: For "text-generation-webui" -> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model) Source versions (and config files) of my models are here: https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be OTHER OPTIONS: - Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor") - If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted. Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers This a "Class 1" model: For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see: [ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ] You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here: [ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ] Optional Enhancement: The following can be used in place of the "system prompt" or "system role" to further enhance the model. It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role". Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented. 
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.

Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)

[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)

Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions. This enhancement WAS NOT used to generate the examples below.

EXAMPLES PROMPTS and OUTPUT:

Examples are created using quant IQ4_XS, "temp=.8" (unless otherwise stated), minimal parameters and "LLAMA3" template. Model has been tested with "temp" from ".1" to "5". Number of experts used is TWO, unless otherwise stated. Below are the least creative outputs, prompt is in BOLD. IMPORTANT: Higher quants / imatrix quants will have much stronger generation - words, sentences, ideas, dialog and general quality. I have included some additional examples at different quant levels for contrast. A "MOE" model "speed" (token per second) will not increase/drop the same way a regular model will on a per quant basis, it will however drop if you engage more experts, as with more experts there is a more processing per token. --- WARNING: NSFW. Vivid prose. Visceral Details. Violence. HORROR. Swearing. UNCENSORED. ---