README.md

---
license: apache-2.0
language:
- en
- fr
- zh
- de
tags:
- creative
- creative writing
- fiction writing
- plot generation
- sub-plot generation
- fiction writing
- story generation
- scene continue
- storytelling
- fiction story
- science fiction
- romance
- all genres
- story
- writing
- vivid prose
- vivid writing
- moe
- mixture of experts
- 64 experts
- 8 active experts
- fiction
- roleplaying
- bfloat16
- rp
- qwen3
- horror
- finetune
- thinking
- reasoning
- qwen3_moe
base_model:
- kalomaze/Qwen3-16B-A3B
pipeline_tag: text-generation
---

(quants group2 (max quants) uploading, one example below, ; more examples to be added...)

<h2>Qwen3-18B-A3B-Stranger-Thoughts-GGUF</h2>

<img src="qwen-18b.jpg" style="float:right; width:300px; height:300px; padding:10px;">

A stranger, yet radically different version of Kalmaze's "Qwen/Qwen3-30B-A3B" with the experts pruned to 64 (from 128)
and then I added 4 layers expanding the model to 18B total parameters.

The goal: slightly alter the model, to address some odd creative thinking and output choices.

Please note that the modifications affect the entire model operation; roughly I adjusted the model to think a little "deeper"
and "ponder" a bit - but this is a very rough description.

I also ran reasoning tests (non-creative) to ensure model was not damaged and roughly matched original model performance.

That being said, reasoning and output generation will be altered regardless of your use case(s).

FOUR example generations below; with example 4 showing a complex prompt and impressive prose that showcases some of the changes in the model.

This is a MOE (Mixture of experts model) with 8 of 64 experts activated by default, which is about 3B parameters.

This allows use of this model (with 8 experts) on both CPU (20-35 T/S) or GPU (90+ T/S) at very good, to extremely fast speeds.

Changing the number of experts used (see below how) will affect both generation speed and generation reasoning/output quality.

You can use this model with as low as four experts activated.

Even the lowest quant - Q2k - will operate very strongly too.

Model is set at:
- 8 Active experts (the default for the org model)
- 40k context (the default for the org model)
- CHATML or Jinja template (embedded OR see Jinja notes below)

QUANTS:

There are two sets of quants, regular and "MAX" (in the filename) with output tensor set at float16 (16 bit - full precision) to enhance performance
including reasoning.

SYSTEM PROMPT:

You may or may not need to set this:

```
You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.
```

CHANGE THE NUMBER OF ACTIVE EXPERTS:

See this document:

https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts

SUGGESTED SETTINGS (creative):

- temp .1 to 1.2 to as high as 2 ; over 2 you may need to "prompt" the model to output after "thinking"
- rep pen 1 to 1.1 (1.05 was tested, see notes below) 
- topk 100, topp .95, min p .05
- rep pen range 64 (default in LMStudio)
- context of 8k min suggested. 

NOTE - REP PEN:
- rep at 1.02 or 1.01 may be better suited for your use cases.
- QWEN3s: rep pen drastically affects both performance and stability.
- change rep pen slowly (ie 1.01, 1.02, 1.03), and regen a few times.

BEST SETTING:

I found the best results during testing (stable thinking, generation, minimal issues) were (this applies to org model 16B-A3B too):
- Jinja Template (corrected one below)
- the system prompt (above)
- rep pen 1.02
- topk 100, topp .95, min p .05
- rep pen range 64
- temp range .6 to 1.2 ; with outside max of 1.8
- context of 8k min 
  
You may find that Qwen's default parameters/samplers also work better for your use case(s) too.

Please refer to the orginal model repo for all default settings, templates, benchmarks, etc.:

https://huggingface.co/Qwen/Qwen3-30B-A3B

For more information on quants, using this model on CPU / GPU, System Prompts, please see this model card:

https://huggingface.co/DavidAU/Qwen3-128k-30B-A3B-NEO-MAX-Imatrix-gguf

SPECIAL NOTE:

As this model is pruned from 128 experts to 64, it may not have the "expert(s)" you need for your use case(s) or
may not perform as well as the full "128" expert version. 

The full 128 expert version (33B) of "Stranger-Thoughts" is located here:

https://huggingface.co/DavidAU/Qwen3-33B-A3B-Stranger-Thoughts-GGUF

<B>NOTE - Jinja Template / Template to Use with this Model:</B>

If you are having issues with Jinja "auto template", use CHATML template.

OR (LMSTUDIO users / option)

Update the Jinja Template (go to this site, template-> copy the "Jinja template" and then paste.) 

[ https://lmstudio.ai/neil/qwen3-thinking ]

OR

copy JINJA source from here:

```
{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
    {%- set index = (messages|length - 1) - loop.index0 %}
    {%- set tool_start = "<tool_response>" %}
    {%- set tool_start_length = tool_start|length %}
    {%- set start_of_message = message.content[:tool_start_length] %}
    {%- set tool_end = "</tool_response>" %}
    {%- set tool_end_length = tool_end|length %}
    {%- set start_pos = (message.content|length) - tool_end_length %}
    {%- if start_pos < 0 %}
        {%- set start_pos = 0 %}
    {%- endif %}
    {%- set end_of_message = message.content[start_pos:] %}
    {%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
        {%- set ns.multi_step_tool = false %}
        {%- set ns.last_query_index = index %}
    {%- endif %}
{%- endfor %}
{%- for message in messages %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {%- set content = message.content %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '</think>' in message.content %}
                {%- set content = (message.content.split('</think>')|last).lstrip('\n') %}
				{%- set reasoning_content = (message.content.split('</think>')|first).rstrip('\n') %}
				{%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\n') %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}
```

---

<B>Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:</B>

In "KoboldCpp" or  "oobabooga/text-generation-webui" or "Silly Tavern" ;

Set the "Smoothing_factor" to 1.5 

: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"

: in text-generation-webui -> parameters -> lower right.

: In Silly Tavern this is called: "Smoothing"


NOTE: For "text-generation-webui" 

-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)

Source versions (and config files) of my models are here:

https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be

OTHER OPTIONS:

- Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")

- If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.

<B>Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>

This a "Class 1" model:

For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]


<b>Optional Enhancement:</B>

The following can be used in place of the "system prompt" or "system role" to further enhance the model.

It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along.
In this case the enhancements do not have as strong effect at using "system prompt" or "system role".

Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.

<PRE>
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.

Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)

[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)

Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
</PRE>

You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation
and scene continue functions.

This enhancement WAS NOT used to generate the examples below.

---

<H2>EXAMPLES</H2>

Standard system prompt, rep pen 1.02, topk 100, topp .95, minp .05, rep pen range 64.

Tested in LMStudio, quant Q4KS, GPU (cpu output will differ slighly).

As this is the mid range quant, expected better results from higher quants and/or with more experts activated to be better.

NOTE: Some formatting lost on copy/paste.

CAUTION: 

Some horror / intense prose.

---

EXAMPLE #1 - temp 1.2

---

<B>
Start a 1000 word scene (vivid, graphic horror in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...
</B>

<P></P>

[[[thinking start]]]

Okay, so I need to start writing a horror scene from Diana's first-person perspective. Let me think about the key elements she wants to convey.

First, the skyscraper swaying. That's a strong image. Maybe she's on the 21st floor, which is a specific detail. The window explodes, so the horror starts with that. I need to build up the horror, the horror elements.

She's in a skyscraper, maybe she's an office worker? Maybe the building is shaking, the window explodes. What's happening around her? Maybe the building is collapsing, or there's an explosion. She's watching the window, so maybe she's inside, looking out, and then the window explodes.

The first-person POV means I need to internalize her thoughts, her feelings. What's her reaction? Fear, horror, maybe confusion. She's a character who's in the middle of a traumatic event.

I need to build up the horror slowly. The skyscraper swaying – maybe the building is shaking, then the window explodes. What's causing it? Maybe an explosion, or something else. Maybe it's the building itself collapsing? Or an external attack? Maybe the sky is dark, and the building is shaking.

I need to make the scene vivid. Use sensory details: the sound of the explosion, the smell of smoke, the feel of the heat, the visual of the window blowing out. Maybe the windows are shattered, and pieces are flying out.

She's the POV character, so her perspective is key. What's going through her mind? Maybe she's trying to understand what's happening, is she trying to run, or is she paralyzed by fear?

Also, the horror aspect – maybe the explosion isn't just a physical thing, but has a supernatural element. Or maybe it's a natural disaster, but she's experiencing it in a horror way. Maybe there's something more eerie, like the building is alive, or the explosion has a malevolent force.

I need to build up the atmosphere. The skyscraper swaying – is it an earthquake? Or maybe the building is being attacked, or there's some supernatural force. Maybe the building is collapsing on itself.

She's watching the window in front of her on the 21st floor explode. So the window is in front of her, maybe she's standing there, looking out, and then the window explodes. What's outside? Maybe the city below, or the sky. Maybe there's a plane coming down, or an explosion.

She's inside the building, and the window explodes. Maybe she's in an office, looking out. Then the window blows out. Maybe she's trying to get away from the blast, or is hit by debris.

I need to make the horror visceral. The horror of the explosion, the sound, the heat, the shock wave, the people screaming outside, the building shaking. Maybe the sky is dark, and there's fire, smoke, explosions, planes falling out of the sky.

I need to make her internal thoughts vivid. Her fear, her panic, her confusion. Maybe she's trying to find a way out, or help, but the building is collapsing around her.

Also, think about the structure of the horror scene. Start with the skyscraper swaying, then the window explodes. Then what happens next. Maybe the building starts to collapse, or more explosions happen. She's trapped inside, trying to escape.

I need to use vivid imagery and sensory details to immerse the reader. The horror is both external and internal. Her emotions are a big part of the horror – the fear, the terror, the horror of what's happening around her.

Maybe she's trying to process what's happening. The window exploding in front of her. What's her reaction? Maybe shock, then panic. She tries to get away, but the building is shaking, windows blowing out.

Also, think about the setting details – the office, the skyscraper, the 21st floor, maybe her name is Diana, so the story starts with "The skyscraper sways..." Then the window explodes, and she's in there.

I need to make the horror graphic, so the description should be intense, visceral. The explosion, the destruction, the panic, the chaos.

Okay, putting it all together now.

[[[thinking end]]]

<p></p>

OUTPUT:

The skyscraper sways, and I think I'm going to throw up. My knees are jelly, but I can't sit down. The building creaks like a wounded beast, groaning under the weight of the sky. I'm on the 21st floor, and the window in front of me explodes.

It doesn't go boooosh—no sound, just a deafening silence, like the world has been sucked into a vacuum. Then the glass explodes outward, a million razor-sharp shards tearing through the air like angel’s teeth. My face is hit—blinding pain, heat, and then the smell of burning glass and something acrid, metallic, sweet. Like blood.

I fall backward, but the floor is already gone. The window, which was a door to the city below, is gone. The sky is black now, and the city below is on fire. I can hear screams. Not just people—something else. A wailing, like a baby with its throat ripped out.

I'm on the floor now, my face stinging where the glass ripped into me, but I don't care. My left eye is blurred, but I can see the building collapsing around me. The walls are shaking, like it's trying to puke itself out of existence. I can smell smoke, and something worse. A rotten sweetness, like meat rotting in the sun.

I try to crawl away from the window, but my fingers are slippery with something—blood, maybe. I don't know. The air is thick with dust and ash. I hear a boom, and the building lurches like a giant man stepping on a pebble.

Then I see it.

Out of the black smoke, something slithers. It's not human. It's long, slick, and black, like oil running down the side of the building. It's not moving—it's hanging, writhing. And then it moves.

It slithers up the side of the building, pulling itself up with some kind of slimy legs. I think I scream, but I can't hear myself over the roar of the wind and the building collapsing around me.

It's getting closer.

I crawl to the far side of the room, pulling my knees up to my chest, and press my hands into the floor. I'm shaking, everywhere. My face burns, my chest hurts, and my legs feel like spaghetti.

The thing is coming up the side of the building now, slithering. It's not a human, it's not a snake—its eyes are black holes, and its mouth is a gash in the flesh. It's not making a sound, but I can feel it. It's thinking.

It wants to get in.

I see people below, running in the street. The sky is orange now, like it's on fire. I can see cars burning, buildings collapsing. People are screaming, falling, and some... some are being pulled up into the smoke, like they're being eaten by the sky itself.

The thing is climbing up the side of the building now, and I think I hear it. Not with my ears, but in my head. A voice, not words, but feeling—open the door.

I don't know what door, but I'm too scared to ask.

I crawl to the door, which is a few feet away. The building is shaking again, and the floor is giving way under me. I'm half-cricling, half-c crawling on my elbows, and I can see the thing now—its black body is like smoke, but with form, like the night itself, writhing and wriggling.

I reach the door. It's open.

And behind it, in the smoke and darkness, I can see shapes.

People, but not people. Their faces are torn, their mouths open in silent screams, and their eyes are black holes.

The thing is right behind me now, and I can feel its breath—hot and sweet, like rotting meat.

I don't know what to do.

I don't know what it wants.

But I do know one thing.

This building isn't collapsing.

It's being eaten from the inside out.

And I'm next.

---

EXAMPLE #2 - temp 1.2

---

<B>
  
</B>

<P></P>

[[[thinking start]]]



[[[thinking end]]]

<p></p>

OUTPUT:


---

EXAMPLE #3 - temp 1.2

---

<B>
  
</B>

<P></P>

[[[thinking start]]]



[[[thinking end]]]

<p></p>

OUTPUT:


---

EXAMPLE #4 - temp 1.2

---

<B>
  
</B>

<P></P>

[[[thinking start]]]



[[[thinking end]]]

<p></p>

OUTPUT: