Cannot find the config file for awq

by SebastianBodza - opened Apr 20

Apr 20

I tried running it with SG Lang and vLLM. Both fail because of the missing config file:
ValueError: Cannot find the config file for awq

If I leave out the model cannot be loaded into 24GB of VRAM . Is it possible to upload the config file?

Owner Apr 20

Thank you for reporting the issue. I have updated the config file. Tested with latest vLLM and it works.

I also recommend the comressed-tensors version instead, because it doesn't have zero points, like the original GGUF release https://huggingface.co/gaunernst/gemma-3-27b-it-qat-compressed-tensors

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment