psyonic-cetacean-20B iMat GGUF

psyonic-cetacean-20B quantized using importance matrix (with love).

Uploading some of the newer quants by request, starting with IQ3.

For more information on the latest importance matrix quants and how they stack up to legacy quantization methods please check out this PR.

Need a different quantization/model? Please open a community post and I'll get back to you - thanks

GGUF

Model size

20B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including InferenceIllusionist/psyonic-cetacean-20B-iMat-GGUF