metadata

language:
  - en
base_model:
  - moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
tags:
  - vllm
  - deepseek_v3
  - deepseek
  - neuralmagic
  - redhat
  - llmcompressor
  - quantized
  - INT4
  - GPTQ
  - conversational
  - custom_code
  - compressed-tensors
  - kimi_k2
license: other
license_name: modified-mit
name: RedHatAI/Kimi-K2-Instruct-quantized.w4a16
description: >-
  This model was obtained by quantizing weights of Kimi-K2-Instruct to INT4 data
  type.
readme: >-
  https://huggingface.co/RedHatAI/Kimi-K2-Instruct-quantized.w4a16/main/README.md
tasks:
  - text-to-text
provider: Moonshot AI
license_link: https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/main/LICENSE

Preliminary version of the model

Evaluations

GSM8k, 5-shot via lm-evaluation-harness

moonshotai/Kimi-K2-Instruct                            = 94.92
RedHatAI/Kimi-K2-Instruct-quantized.w4a16 (this model) = 94.84

More evals coming very soon...