I have no idea what I’m doing… if this causes the apocalypse someone please let me know.

L3-Arcania-4x8b 2.5bpw h8 EXL2

Includes measurement.json file for further quantization

Original Model: https://huggingface.co/Steelskull/L3-Arcania-4x8b

Original Model Card

L3-Arcania-4x8b Data Card

Now that the cute anime girl has your attention.

Creator: SteelSkull

About L3-Arcania-4x8b: A Mixture of Experts model designed for general assistance, storytelling, roleplay, and ERP.

Integrates models from notable sources for enhanced performance in diverse tasks.

This model is based off of the work ive done on Umbra v1-v3 basically the gates are trained off of Keywords that direct the gates but not limit as much as a full prompt would. My goal is Quality not quantity

Source Models:

Quants:

Recommended: (Thanks to @Mradermacher!, please send them likes!)

L3-Arcania-4x8b-GGUF (all GGUFs)

L3-Arcania-4x8b-i1-GGUF (i Quant GGUFs)

My Quants: (they work, just not many choices)

SteelQuants/L3-Arcania-4x8b-Q4_K_M-GGUF

SteelQuants/L3-Arcania-4x8b-Q5_K_M-GGUF

Config:

Recommended Prompt Format: [Llama 3]

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{{prompt}}<|eot_id|>{{history}}<|start_header_id|>{{char}}<|end_header_id|>

Model Config:

MODEL_NAME = "L3-Arcania-4x8b"
base_model: meta-llama/Meta-Llama-3-8B-Instruct
gate_mode: hidden 
dtype: bfloat16
experts:
  - source_model: Sao10K/L3-Solana-8B-v1
  - source_model: dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5
  - source_model: NeverSleep/Llama-3-Lumimaid-8B-v0.1
  - source_model: cgato/L3-TheSpice-8b-v0.1.3

L3-Arcania-4x8b combines the strengths of multiple models to deliver a well-rounded, capable assistant. It excels at general tasks, storytelling, roleplay, and even more mature content.

The base model, Meta-Llama-3-8B-Instruct, provides a solid foundation. The expert models then enhance specific capabilities:

L3-Solana-8B-v1 adds generalist knowledge and the ability to handle a wide range of topics, including NSFW content.
opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 strengthens storytelling, roleplay, and long-form writing abilities.
Llama-3-Lumimaid-8B-v0.1 introduces expertise in romantic, flirtatious, and explicit interactions.
L3-TheSpice-8b-v0.1.3 ensures the model remains focused, tailored, and high-quality.

The positive and negative prompts guide each expert's influence, resulting in a model that is versatile yet refined, capable of both general assistance and more specialized, mature interactions.