Possibility for a 2.85BPW quant?
#1
by
Adzeiros
- opened
Hey,
First off, thank you for making an EXL3 version 🙏
I only have 48GB of VRAM so I think my max quant for a 123B model is 2.85bpw. If you could create one at that quant that would be absolutely amazing!
Either way, tysm
eta, 3h45m
Thank you!
Awesome thank you!
gecfdo
changed discussion status to
closed