metadata

license: apache-2.0
language:
  - zh
  - en

Chinese-Mixtral-Instruct-GGUF

This repository contains the GGUF-v3 models (llama.cpp compatible) for Chinese-Mixtral-Instruct (chat/instruction model).

Performance

Metric: PPL, lower is better

Quant	PPL
IQ2_XXS	-
IQ2_XS	-
Q2_K	4.5758 +/- 0.03959
Q3_K	4.5563 +/- 0.04126
Q4_0	3.9757 +/- 0.03455
Q4_K	3.9265 +/- 0.03407
Q5_0	3.9167 +/- 0.03399
Q5_K	3.9232 +/- 0.03403
Q6_K	3.9242 +/- 0.03415
Q8_0	3.9159 +/- 0.03402
F16	x

Due to the file size limitation, for F16 model, please use cat command to concatenate all parts into a single file. You must concatenate these parts in order.

Others

For Hugging Face version, please see: https://huggingface.co/hfl/chinese-mixtral-instruct

Please refer to https://github.com/ymcui/Chinese-Mixtral/ for more details.