metadata
base_model: Qwen/Qwen2-VL-72B-Instruct
license: other
license_name: tongyi-qianwen
license_link: https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct/blob/main/LICENSE
model_creator: Qwen
model_name: Qwen2-VL-72B-Instruct
quantized_by: Second State Inc.
language:
- en
pipeline_tag: image-text-to-text
tags:
- multimodal
library_name: transformers
Qwen2-VL-72B-Instruct-GGUF
Original Model
Run with LlamaEdge
- LlamaEdge version: coming soon
Quantized GGUF Models
| Name | Quant method | Bits | Size | Use case |
|---|---|---|---|---|
| Qwen2-VL-72B-Instruct-Q3_K_M.gguf | Q3_K_M | 3 | 37.7 GB | very small, high quality loss |
| Qwen2-VL-72B-Instruct-Q4_0.gguf | Q4_0 | 4 | 41.2 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
| Qwen2-VL-72B-Instruct-Q5_K_M-00001-of-00002.gguf | Q5_K_M | 5 | 29.9 GB | large, very low quality loss - recommended |
| Qwen2-VL-72B-Instruct-Q5_K_M-00002-of-00002.gguf | Q5_K_M | 5 | 24.5 GB | large, very low quality loss - recommended |
| Qwen2-VL-72B-Instruct-vision-encoder.gguf | f16 | 16 | 2.8 GB |
Quantized with llama.cpp b4372