"Should run on 12 GB of VRAM cards..."
#1
by
CulturedMan
- opened
I tried loading this on my 3060 with 4096 context with cache_8bit on, and about 2.5 GB is going to shared memory. I think that 12 GB VRAM cards need 3.0bpw to fit it all in the card.
Ok, good to know, ill upload new quant on weekend.
Thanks, that's greatly appreciated!