GGUF?

#2
by Alastar-Smith - opened

Is there a way that we can use in LMStudio as GGUF Qs?

I will try out to make it work, maybe the recent changes in llama.cpp make it possible (;

bad news currently there is more work needed

I have managed to create a Q8_0 gguf and a mmproj gguf, now i need to test inference

Is there a way that we can use in LMStudio as GGUF Qs?

are you currently online?

Sign up or log in to comment