L3.3-Shakudo-70b - EXL3 3.0bpw H6
This is a 3bpw EXL3 quant of Steelskull/L3.3-Shakudo-70b
This quant was made using exllamav3-0.0.5 with '--cal_cols 4096' (instead of default 2048) which in my experience improves quant quality a bit
3bpw fits in 32GB VRAM on Windows with around 18-20k Q8 context
I tested this quant shortly in some random RPs (including ones over 8k and 16k context) and it seems to work fine
Prompt Templates
Uses Llama 3 Instruct format. Supports thinking with "<thinking>" prefill in assistant response.
Original readme below
this is designed for Dark mode
L3.3-Shakudo-70b
⚡ Top Sponsors
🏆 Top Supporters
If I forgot you please let me know, ko-fi doesent let me track it easily
🤝 Valued Partners
Model Information
L3.3-Shakudo-70b
Model Composition
-
Final Merge: L3.3-Shakudo-70b
-
Model 1: L3.3-M1-Hydrargyrum-70B
Model Creation Process
L3.3-Shakudo-70b is the result of a multi-stage merging process by Steelskull, designed to create a powerful and creative roleplaying model with a unique flavor. The creation process involved several advanced merging techniques, including weight twisting, to achieve its distinct characteristics.
Stage 1: The Cognitive Foundation & Weight Twisting
The process began by creating a cognitive and tool-use focused base model, L3.3-Cogmoblated-70B. This was achieved through a `model_stock` merge of several models known for their reasoning and instruction-following capabilities. This base was built upon `nbeerbower/Llama-3.1-Nemotron-lorablated-70B`, a model intentionally "ablated" to skew refusal behaviors. This technique, known as weight twisting, helps the final model adopt more desirable response patterns by building upon a foundation that is already aligned against common refusal patterns.
Stage 2: The Twin Hydrargyrum - Flavor and Depth
Two distinct models were then created from the Cogmoblated base:
- L3.3-M1-Hydrargyrum-70B: This model was merged using `SCE`, a technique that enhances creative writing and prose style, giving the model its unique "flavor." The Top_K for this merge were set at 0.22 .
- L3.3-M2-Hydrargyrum-70B: This model was created using a `Della_Linear` merge, which focuses on integrating the "depth" of various roleplaying and narrative models. The settings for this merge were set at: (lambda: 1.1) (weight: 0.2) (density: 0.7) (epsilon: 0.2)
Final Stage: Shakudo
The final model, L3.3-Shakudo-70b, was created by merging the two Hydrargyrum variants using a 50/50 `nuslerp`. This final step combines the rich, creative prose (flavor) from the SCE merge with the strong roleplaying capabilities (depth) from the Della_Linear merge, resulting in a model with a distinct and refined narrative voice.
A special thank you to Nectar.ai for their generous support of the open-source community and my projects.
Additionally, a heartfelt thanks to all the Ko-fi supporters who have contributed, your generosity is deeply appreciated and helps keep this work going and the Pods spinning.
-
Recommended Sampler Settings
Good Starting Templates & Prompts
Support & Community:
- Downloads last month
- 12
Model tree for DeusImperator/L3.3-Shakudo-70b_exl3_3.0bpw_H6
Base model
meta-llama/Llama-3.1-70B