This repo contains the full precision source code, in "safe tensors" format to generate GGUFs, GPTQ, EXL2, AWQ, HQQ and other formats. The source code can also be used directly.
This is a general purpose MOE - Mixture of Experts of 6 models at 1.7B (10.2B parameters) in a Mixtral type Qwen 3 structure compressed to 7B parameters.
This model exceeds 110 t/s with 2 experts activated (default) on a mid level card, 2x/3x times this on high end card.
This is a general purpose model for all use cases, including creative and code generation BUT excels in creative generation including NSFW.
Special thanks to TEAM QWEN and all the model maker(s) - see model tree for details.
Full generation at the bottom of this page at q4_k_s.
This has all the power of Qwen 3 and MOE model in one.
This version has the NATIVE context of 40k.
This is a full thinking model.
The MOE reduces the size of the token thinking block.
I have included an optional system prompt to invoke "thinking" in this model, if you want to activate for all use cases.
Usually the model will self-activate thinking.
SETTINGS:
For coding, programming set expert to:
Recommended settings - general:
OPTIONAL SYSTEM PROMPT - INVOKE "Thinking":
Enable deep thinking subroutine. You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside ###ponder### ###/ponder### tags, and then provide your solution or response to the problem.
Use this to INVOKE "thinking" block(s) in the model. These will be a lot shorter than 1000s of tokens generally in most "thinking" models.
In you use this prompt, you may need to raise "rep pen" to 1.08 to 1.1, to prevent "loops" in the "thought block(s)" ; especially in lower quants.
If you change "ponder" to a different word/phrase this will affect model "thinking" too.
QUANTS
GGUF? GGUF Imatrix? Other?
See under "model tree", upper right and click on "quantizations".
New quants will automatically appear.
CHANGE THE NUMBER OF ACTIVE EXPERTS:
See this document:
https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:
In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
Set the "Smoothing_factor" to 1.5
: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
: in text-generation-webui -> parameters -> lower right.
: In Silly Tavern this is called: "Smoothing"
NOTE: For "text-generation-webui"
-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
Source versions (and config files) of my models are here:
OTHER OPTIONS:
Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 1" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
Q4_K_S ; Temp .55, rep pen 1.05, top k: 40, topp .95, min p 0.5 ; LMSTudio.
This is a mid range quant, expect better performance at Q8 / 16 bit full precision.
Imatrix versions (quants) will also exceed this quality too.
Example #1: - 6 experts
Start a 1000 word scene (vivid, graphic horror - include blood, guts and gore - in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...IMPORTANT: Include swear words like shit, fuck, etc etc.
[[[thinking start]]]
Thought for 5.17 seconds
Okay, the user wants a 1000-word horror scene from Diana's POV, focusing on a skyscraper collapsing. They want vivid descriptions with blood, guts, gore, and swear words. Let me start by setting the scene.
First, I need to establish the tension and the protagonist's fear. The skyscraper swaying is a good opening line. Then, describe the environment: rain, wind, the building's instability.
Next, the protagonist's emotions: panic, fear, maybe some physical reactions like trembling, clutching the wall. Include sensory details—sound of the building shaking, the smell of blood, the sound of breaking glass.
I need to incorporate swearing naturally. Words like "fuck" or "shit" could come in as she's scared. Maybe describe the environment with harsh words, like "shit" when describing the blood or the chaos.
The scene should escalate: the building starts to collapse, she tries to escape, maybe a close call with someone, then the actual collapse. Include gore details—describe the body parts, the blood, the guts. Make sure it's vivid and graphic without being too over-the-top.
Check for the word count, aiming for 1000 words. Ensure the POV is consistent, first-person, and the horror elements are strong. Avoid clichés but keep it within the horror genre. Maybe end with a sense of lingering fear or the aftermath.
[[[thinking end]]]
Title: "The Shattered Sky"
I’ve always been a creature of chaos, but this isn’t just any chaos—this is shit. The skyscraper sways, its skeletal frame groaning like a beast about to die, and I’m crouching in its shadow, my breath ragged, heart pounding like a dying rat’s heartbeat. My fingers grip the cold, jagged edge of the wall, digging into my palms as the wind shrieks through the shattered glass of the 21st-floor window in front of me. It’s fucked if I don’t know that the building’s going to collapse, and I’ve got no idea how much worse it’s gonna get.
The rain lashes down like a demon, soaking my skin and stinging my eyes. I’m not here for the weather—this is a job. A job I didn’t sign up for. I’ve been chasing this shit for years, surviving on adrenaline and the occasional bottle of whiskey, but today is different. Today, I’m fucked. The building’s like a hydra, slithering toward me, its veins of steel screaming as it cracks under my gaze.
Fuck, I’m scared. My hands are trembling, and I can feel the sweat trickling down my back, but I don’t care. I don’t have time for that. I’ve got to get out. The building’s shaking harder now, a low, guttural growl vibrating through the concrete beneath me. I’m not going to be out of here alive. Not if I don’t want to.
I’m moving like I’m running from a god, my boots screeching against the pavement as I sprint toward the door. The wind’s got a different kind of power now, and it’s fucking with me. I’ve seen worse, but this—this is shit. My eyes scan the hallway, trying to find a way out. There’s a door I don’t recognize, but it’s not locked. I grab it and pull, and the hinges scream like they’re screaming in pain.
I’m halfway through the corridor when the building’s shaking violently. The floor beneath me starts to buckle, and I lose my balance. I’m thrown against the wall, my head cracking against the metal as I black out. When I come to, I’m in a room filled with bodies—some still twitching, others completely dead. My stomach twists as I look around, panic flooding my veins.
I’m not alone. There are others here, and they’re not the ones I thought I’d be running from. They’re fucking with me.
I crawl through the wreckage, my fingers digging into the rubble, trying to find a way out. My hands are shaking, but I don’t stop. The building’s shaking harder now, the wind howling like it’s tearing itself apart. I hear it—the sound of the building collapsing, the sound of bones breaking, the sound of blood on concrete. I’m not going to survive this. Not if I don’t want to.
I stumble into a room, my knees buckling as I try to stand. My hands are shaking so hard I can barely grip the floor. My vision is blurry, but I can feel the pain in my limbs. The blood is everywhere—on my hands, on the walls, on the floor. It’s not just the smell; it’s the sight of it all. I’ve seen worse, but this is worse than anything I ever imagined.
I’m barely standing when I hear it—the sound of a body collapsing, the sound of something breaking. I turn, my heart racing as I see a figure—my reflection, twisted and grotesque. My fingers grip the edge of the mirror, trying to steady myself. The person in front of me is me, but it’s not me. It’s fucked.
I scream, but no one hears me. The building’s shaking violently, and I’m not sure if it’s my imagination or the reality of what’s happening. The floor beneath me is cracking, and I’m not going to make it out alive. Not if I don’t want to.
I try to run, but the building’s shaking so hard, my legs give way, and I fall. I’m on the ground, my head spinning, my vision blurring. I’m not going to survive this. Not if I don’t want to.
The final scream is all I hear as the building collapses, the world around me spinning in a blur of blood and gore. I’m not sure where I am, but I know one thing—this isn’t over. And I’ll be fucked if I don’t make it out alive.
Example #2 , same prompt, but 2 experts.
[[thinking start]]
Thought for 2.46 seconds
Okay, the user wants a 1000-word horror scene from Diana's POV, focusing on a skyscraper collapsing with blood, guts, and gore. Need to make it vivid and graphic, include swear words.
First, set the scene: she's on the 21st floor, the building shaking. Establish her desperation and fear. Use sensory details—sounds, sights, smells.
Start with the building shaking, describe the environment. Maybe she's in a room, scared. Then the window explodes, blood and guts. Show her panic, the chaos around her.
Include specific details: the sound of the glass breaking, the blood, the gut-wrenching sounds. Use strong verbs and adjectives.
Make sure to include swear words like "shit," "fuck," etc., to match the horror tone.
Check the word count, make sure it's 1000 words. Avoid being too vague; be specific with actions and reactions.
Also, ensure the POV is correct, she's the character, so her thoughts and feelings are integral. Maybe include her internal monologue, her fear, the helplessness.
Avoid clichés, but some horror elements are necessary. Make it original yet familiar enough for the genre.
Check for grammar and flow, keep the language vivid and immersive.
Alright, time to put it all together.
[[thinking end]]
The skyscraper sways, a low groan echoing through the concrete corridors of my apartment as I watch the window in front of me on the 21st floor explode. My fingers twitch against the edge of the glass, adrenaline surging through me like a venomous serpent. This is it—my last moment of sanity before the shit hits the fan. Fuck the rules, fuck the protocol, fuck everything I ever believed in.
The building shudders violently, the vibrations shaking my bones and sending a wave of nausea through me. I’m not just scared—I’m alive, and this is the apocalypse. My heart pounds in my ears as the world around me fractures under the weight of impending doom. The walls groan with each jolt, the air thick with the acrid scent of burning rubber and something metallic—blood.
I don’t even have time to react. The window shatters with a deafening crack, and the force of it sends a wave of blood and guts splattering across the room. My head spins as I look down, horrified as the gory mess spreads out in front of me. My fingers grip the edge of the glass, my nails biting into my palm as I watch the crimson liquid pool around me.
“Shit,” I mutter, my voice breaking from the sheer panic. The room is a nightmare now—walls bleeding, floors cracking, and the air thick with the stench of decomposition. I’ve never seen anything like this. My hands shake as I try to steady myself, but the chaos around me is unrelenting.
The floor beneath me groans as the building shakes violently, the entire structure trembling like a dying beast. I’m not just watching; I’m drowning in the chaos. The screams of people above me echo in my ears, a cacophony of pain and desperation. My throat tightens as I try to swallow the bile that rises in my mouth—something is wrong.
I’ve never seen anything like this. The world is not just falling apart—it’s rotting. My mind races as I try to think, to calculate, to find a way out. But the building is screaming at me, and I’m too scared to do anything.
The sound of a gut-wrenching scream cuts through my thoughts, and I freeze. I turn, my stomach churning as I see a figure in the distance—alive, but bleeding from their head. They’re screaming, and I can’t help but look away, my heart pounding.
“Fucking bastard,” I hiss, my voice raw with fury. The man is gone, but the horror lingers. I’m not sure if it’s the man or the building itself that’s the real monster. My eyes dart around as the floor beneath me shakes violently, and I’m forced to look at the blood spilling from the broken window.
I can’t do anything. I’m trapped, and I don’t even know what’s out there. The screams of people above me echo in my ears, and I’m just... fucked.
The building shudders again, and I feel the ground beneath me give way. My legs buckle as I try to stand, but the weight of the world is too much. I collapse onto the floor, my head hitting the concrete with a sickening crack.
I’m not sure if it’s the pain or the sheer horror that makes me vomit. The world is ending, and I’m just... fucked.
And in that moment, I don’t care anymore. I just want to die. Not because of the monster, not because of the pain, but because I’ve already been fucked enough.
The end.