metadata
license: apache-2.0
base_model:
- Qwen/Qwen2.5-72B-Instruct
tags:
- autoround
This is Qwen/Qwen2.5-72B-Instruct quantized with AutoRound in 2-bit (symmetric + gptq format) with a group size of 64 and calibration samples of 4096 tokens. The model has been created, tested, and evaluated by The Kaitchup. The model is compatible with vLLM and Transformers.
More details in this article: Accurate 2-bit Quantization: Run Massive LLMs on a Single Consumer GPU
- Developed by: The Kaitchup
- License: Apache 2.0 license
How to Support My Work
Subscribe to The Kaitchup. This helps me a lot to continue quantizing and evaluating models for free.