hfl
/

hfl-rc commited on
Commit
3f4304b
1 Parent(s): 2a306f4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -9,6 +9,24 @@ language:
9
 
10
  This repository contains GGUF-v3 version (llama.cpp compatible) of **Chinese-LLaMA-2-7B-64K**, which is tuned on Chinese-LLaMA-2-7B with **YaRN method**.
11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  For full model in HuggingFace format, please see: https://huggingface.co/hfl/chinese-llama-2-7b-64k
13
 
14
  Please refer to [https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/) for more details.
 
9
 
10
  This repository contains GGUF-v3 version (llama.cpp compatible) of **Chinese-LLaMA-2-7B-64K**, which is tuned on Chinese-LLaMA-2-7B with **YaRN method**.
11
 
12
+ ## Performance
13
+
14
+ Metric: PPL, lower is better
15
+
16
+ | Quant | original | imatrix (`-im`) |
17
+ |-----|------|------|
18
+ | Q2_K | 11.5424 +/- 0.24106 | 12.1599 +/- 0.26050 |
19
+ | Q3_K | 10.0152 +/- 0.21296 | 9.9269 +/- 0.21335 |
20
+ | Q4_K | 9.7687 +/- 0.21133 | 9.7239 +/- 0.20999 |
21
+ | Q5_K | 9.6229 +/- 0.20829 | 9.5673 +/- 0.20675 |
22
+ | Q6_K | 9.5996 +/- 0.20816 | 9.5753 +/- 0.20734 |
23
+
24
+ *The model with `-im` suffix is generated with important matrix, which has generally better performance (not always though).*
25
+
26
+
27
+ ## Others
28
+
29
+
30
  For full model in HuggingFace format, please see: https://huggingface.co/hfl/chinese-llama-2-7b-64k
31
 
32
  Please refer to [https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/) for more details.