    # Brianpuze/Qwen2.5-0.5B-Q4_K_M-Q3_K_M-GGUF-TEST
    This repo contains GGUF quantized versions of [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) using llama.cpp.

    ## Quantized Versions:
    - qwen2.5-0.5b-q4_k_m.gguf

qwen2.5-0.5b-q3_k_m.gguf
```
  ## Run with llama.cpp
```

llama-cli --hf-repo Brianpuze/Qwen2.5-0.5B-Q4_K_M-Q3_K_M-GGUF-TEST --hf-file qwen2.5-0.5b-q4_k_m.gguf -p "The meaning of life is"

    (Replace filename to use other variants.)

Downloads last month: 2

GGUF

Model size

494M params

Architecture

qwen2

Hardware compatibility

3-bit

4-bit

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Brianpuze/Qwen2.5-0.5B-Q4_K_M-Q3_K_M-GGUF-TEST

Base model

Qwen/Qwen2.5-0.5B

Quantized

(60)

this model