Undi95/BagelMix-8x7B 3.5bpw
Exllama quant of Undi95/BagelMix-8x7B
You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)
Prompt format:
ChatML? maybe? Unclear
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
Contact
Kooten on discord.
- Downloads last month
- 2
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.