|
|
--- |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- moonshotai/Kimi-K2-Instruct |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- vllm |
|
|
- deepseek_v3 |
|
|
- deepseek |
|
|
- neuralmagic |
|
|
- redhat |
|
|
- llmcompressor |
|
|
- quantized |
|
|
- INT4 |
|
|
- GPTQ |
|
|
- conversational |
|
|
- custom_code |
|
|
- compressed-tensors |
|
|
- kimi_k2 |
|
|
license: other |
|
|
license_name: modified-mit |
|
|
name: RedHatAI/Kimi-K2-Instruct-quantized.w4a16 |
|
|
description: This model was obtained by quantizing weights of Kimi-K2-Instruct to INT4 data type. |
|
|
readme: https://huggingface.co/RedHatAI/Kimi-K2-Instruct-quantized.w4a16/main/README.md |
|
|
tasks: |
|
|
- text-to-text |
|
|
provider: Moonshot AI |
|
|
license_link: https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/main/LICENSE |
|
|
--- |
|
|
|
|
|
# Preliminary version of the model |
|
|
|
|
|
## Evaluations |
|
|
|
|
|
- GSM8k, 5-shot via lm-evaluation-harness |
|
|
``` |
|
|
moonshotai/Kimi-K2-Instruct = 94.92 |
|
|
RedHatAI/Kimi-K2-Instruct-quantized.w4a16 (this model) = 94.84 |
|
|
``` |
|
|
|
|
|
More evals coming very soon... |
|
|
|
