Anubis-70B-v1.1 - EXL3 3.0bpw H6

This is a 3bpw EXL3 quant of TheDrummer/Anubis-70B-v1.1

This quant was made using exllamav3-0.0.4 with '--cal_cols 4096' (instead of default 2048) which in my experience improves quant quality a bit

3bpw fits in 32GB VRAM on Windows with around 18-20k Q8 context (tested in tabbyAPI)

I tested this quant shortly in some random RPs (including ones over 8k and 16k context) and it seems to work fine

Prompt Templates

Uses Llama 3 Instruct format.

Original readme below

Join our Discord! https://discord.gg/BeaverAI

More than 6000 members strong 💪 A hub for users and makers alike!

Live in OpenRouter! (Powered by Parasail.io)

Drummer proudly presents...

Anubis 70B v1.1 💿 - Shimmer Edition

That's how the madness of the world tries to colonize you: from the outside in, forcing you to live in its reality.

Supported Chat Templates

Llama 3 Chat Template

Description

A follow up to Anubis 70B v1.0 but with two main strengths: character adherence and unalignment.

This is not a minor update to Anubis. It is a totally different beast.

The model does a fantastic job portraying my various characters without fail, adhering to them in such a refreshing and pleasing degree with their dialogue and mannerisms, while also being able to impart a very nice and fresh style that doesn't feel like any other L3.3 models.

I do think it's a solid improvement though, like it nails characters.

It feels fresh. I am quite impressed on how it picked up on and empasized subtle details I have not seen other models do in one of my historically accurate character cards.

Anubis v1.1 is in my main model rotation now, I really like it! -Tarek

Special Thanks

Thank you once again Geechan for leading with the finishing touch!
Thank you to each and everyone who donated and subscribed in Patreon and Ko-Fi to make our venture a little bit easier.
Subscribe to my Patreon!

DeusImperator
/

Anubis-70B-v1.1_exl3_3.0bpw_H6