Hugging Face's logo

medmekk
/
Qwen2.5-0.5B-Instruct-bnb-4bit

Safetensors
qwen2
4-bit precision
bitsandbytes
Model card Files Community
Qwen2.5-0.5B-Instruct-bnb-4bit / README.md
medmekk's picture
medmekk HF Staff
Upload folder using huggingface_hub
b331341 verified
preview code
|
raw
history blame contribute delete
718 Bytes
metadata
base_model:
  - medmekk/Qwen2.5-0.5B-Instruct-bnb-4bit

medmekk/Qwen2.5-0.5B-Instruct-bnb-4bit (Quantized)

Description

This model is a quantized version of the original model medmekk/Qwen2.5-0.5B-Instruct-bnb-4bit. It has been quantized using int4 quantization with bitsandbytes.

Quantization Details

  • Quantization Type: int4
  • bnb_4bit_quant_type: nf4
  • bnb_4bit_use_double_quant: True
  • bnb_4bit_compute_dtype: bfloat16
  • bnb_4bit_quant_storage: uint8

Usage

You can use this model in your applications by loading it directly from the Hugging Face Hub:

from transformers import AutoModel

model = AutoModel.from_pretrained("medmekk/Qwen2.5-0.5B-Instruct-bnb-4bit")