Forgotten-Abomination-12B-v4.0-EXL2

This repository contains exl2 quants of Forgotten-Abomination-12B-v4.0, each in respective branch.

Built-in(default) calibration dataset is used.

Download:

To download, open directory you want to download to in your shell/CLI, then use huggingface-cli:

6bpw:

huggingface-cli download riveRiPH/Forgotten-Abomination-12B-v4.0-EXL2 --local-dir Forgotten-Abomination-12B-v4.0-6bpw-h8-exl2 --local-dir-use-symlinks False --revision 6bpw

5bpw:

huggingface-cli download riveRiPH/Forgotten-Abomination-12B-v4.0-EXL2 --local-dir Forgotten-Abomination-12B-v4.0-5bpw-h8-exl2 --local-dir-use-symlinks False --revision 5bpw

4bpw:

huggingface-cli download riveRiPH/Forgotten-Abomination-12B-v4.0-EXL2 --local-dir Forgotten-Abomination-12B-v4.0-4bpw-h8-exl2 --local-dir-use-symlinks False --revision 4bpw