Spaces:

ggml-org
/

gguf-my-repo

Running on A10G

GPU is not used?

#84

by Clausss - opened Jun 11

Jun 11

gpu is not used during quanization 8 bit of llama 8b instruct only ram?

Jun 11

Hey! In order to use imatrix quantization you have to check the box below the text before submitting in order to offload to GPU :)

Jun 11

thanks for quick answer!

Clausss changed discussion status to closed Jun 11

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment