File size: 924 Bytes
ecbdad4 5bafe6f ecbdad4 5bafe6f ecbdad4 5bafe6f ecbdad4 5242783 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
language:
- en
base_model:
- moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
tags:
- vllm
- deepseek_v3
- deepseek
- neuralmagic
- redhat
- llmcompressor
- quantized
- INT4
- GPTQ
- conversational
- custom_code
- compressed-tensors
- kimi_k2
license: other
license_name: modified-mit
name: RedHatAI/Kimi-K2-Instruct-quantized.w4a16
description: This model was obtained by quantizing weights of Kimi-K2-Instruct to INT4 data type.
readme: https://huggingface.co/RedHatAI/Kimi-K2-Instruct-quantized.w4a16/main/README.md
tasks:
- text-to-text
provider: Moonshot AI
license_link: https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/main/LICENSE
---
# Preliminary version of the model
## Evaluations
- GSM8k, 5-shot via lm-evaluation-harness
```
moonshotai/Kimi-K2-Instruct = 94.92
RedHatAI/Kimi-K2-Instruct-quantized.w4a16 (this model) = 94.84
```
More evals coming very soon...
|