google

/

gemma-3-12b-it-qat-q4_0-gguf

like 147

Follow

Google 17.2k

Hi @chenbridge ,

try following

• Copy your Ollama SSH key, you can do so via: cat ~/.ollama/id_ed25519.pub | pbcopy
• Add the corresponding key to your Hugging Face account by going to your account settings and clicking on Add new SSH key.
• That’s it! You can now run private GGUFs from the Hugging Face Hub: ollama run hf.co/google/gemma-3-12b-it-qat-q4_0-gguf
  reference : https://github.com/ollama/ollama/issues/8045

Kindly try and let us know if you have any concerns. Thank you.

{'id': 'chatcmpl-5dde1400-febd-48d5-94f1-de1e86076abf',
'object': 'chat.completion',
'created': 1747907389,
'model': .../.cache/huggingface/hub/models--google--gemma-3-12b-it-qat-q4_0-gguf/snapshots/7929e34ac7c459bbfd1f38d7358e6839f204578b/./gemma-3-12b-it-q4_0.gguf',
'choices': [{'index': 0,
'message': {'role': 'assistant',
'content': 'Please provide me with the image you would like me to describe! I need to see the image to give you a one-sentence description.\n'},
'logprobs': None,
'finish_reason': 'stop'}],
'usage': {'prompt_tokens': 18, 'completion_tokens': 29, 'total_tokens': 47}}

running this , i get the error above ?
image1 = '7000_images/20190111_003017822/image_p2-27.png'

image1 = Image.open(image1°
llm.create_chat_completion(
messages = [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image in one sentence."
},
{
"type": "image",
"image": image1
}
]
}
]
)