Quantized Qwen2.5-pyEvaluator-3B-GGUF-Q4_K_M Model (GGUF Format)

This is a quantized and finetune version of a Qwen2.5-Coder-3B model, converted to GGUF format and quantized to Q4_K_M using llama.cpp. This is fine tune on the specific dataset to evaluate only python code. The model is optimized for efficient inference on CPU-based systems.

Model Details

• Model Name: Qwen2.5-pyEvaluator-3B-GGUF-Q4_K_M
• Base Model: Qwen2.5-Coder-3B
• Quantization Type: Q4_K_M
• Framework: llama.cpp
• Model Size: 1.79 GB
• Author: Moiz2517

Quantization Details

This model was quantized using llama.cpp with the following settings:

• Quantization method: Q4_K_M
• Bits: 4-bit

The quantization process reduces the model size while maintaining a good balance between performance and accuracy.

Usage

Using llama.cpp

You can load and run this model directly using ollama or hugging face. Here's an example:

ollama run hf.co/Moiz2517/Qwen2.5-pyEvaluator-3B-GGUF-Q4_K_M