Minimum VRAM
Amazing work guys I tried on hugging face spaces an really sounds cool... now ... tried to run locally on docker container on my 3060 12gb... OOM... what is the minimum amount of Vram for this models?
Can't run it myself cuz I don't even have enough RAM but their HF space uses ~41GB VRAM so I guess you should be good with 42GB.
Well if thats true ... its a no-no for me ... gonna have to wait for the quantized version lol... too bad
It is using 17476MiB for me for reference
It can definitely run on a 24GB consumer GPU (I tested), not sure about smaller
Yes, I am using a 4090
For me it runs on my 3060 and consumes 11.7GB / 12GB vram. Maybe it runs in half precision out of the box? Because i don't use any arguments other then --port to start. Using windows 11, nvidia driver version 576.02 and cuda 12.8.