Text generation web UI, getting error message
#27
by
JB0007
- opened
Hi!
I got this error message and I don't know what to this. Sorry, I'm new to this and I don't know what I'm doing.
ImportError: cannot import name 'llama_inference_offload' from 'gptq_for_llama' (C:\Users\jaren\Downloads\text-generation-webui-main\installer_files\env\Lib\site-packages\gptq_for_llama_init_.py)
Best,
Jarebear
+1
Me too
Same here, I am using MAc and other error is related to CUDA, I tried to uninstall and reinstall but in vain. any help is appreciated
@Truthman48 well gptq is not good for mac users. It’s extremely slow and only uses cpu on Mac.
You have to use gguf models and llama cpp as the loader as that’s the fastest and highest quality for mac