Update README.md
Browse files
README.md
CHANGED
@@ -20,43 +20,28 @@ tags:
|
|
20 |
- static
|
21 |
---
|
22 |
|
23 |
-
|
24 |
-
<div style="border: 2px solid #c4382d; border-left-width: 8px; padding: 16px 32px; border-radius: 0 10px 10px 0;">
|
25 |
-
<h3>Known Issue:</h3>
|
26 |
-
|
27 |
-
Initial version of <a href="https://huggingface.co/mistralai/Codestral-22B-v0.1">mistralai/Codestral-22B-v0.1</a> had a missing tokens for FIM (Fill In the Middle) - see <a href="https://huggingface.co/mistralai/Codestral-22B-v0.1/discussions/10#665856e3d3e05bb21be16140">discussion</a>
|
28 |
-
|
29 |
-
<h3>Fixed here:</h3>
|
30 |
-
<ul>
|
31 |
-
<li><a href="https://huggingface.co/legraphista/Codestral-22B-v0.1-hf-FIM-fix-IMat-GGUF">legraphista/Codestral-22B-v0.1-hf-FIM-fix-IMat-GGUF</a> </li>
|
32 |
-
<li><a href="https://huggingface.co/legraphista/Codestral-22B-v0.1-hf-FIM-fix">legraphista/Codestral-22B-v0.1-hf-FIM-fix</a></li>
|
33 |
-
</ul>
|
34 |
-
</div>
|
35 |
-
|
36 |
-
<br>
|
37 |
-
|
38 |
# Codestral-22B-v0.1-IMat-GGUF
|
39 |
_Llama.cpp imatrix quantization of mistralai/Codestral-22B-v0.1 (bullerwins/Codestral-22B-v0.1-hf)_
|
40 |
|
41 |
Original model: [mistralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1)
|
42 |
-
Quantized HF Model: [
|
43 |
Original dtype: `BF16` (`bfloat16`)
|
44 |
-
Quantized by: llama.cpp [
|
45 |
IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
|
46 |
|
47 |
-
- [
|
48 |
-
- [
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
- [
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
|
61 |
---
|
62 |
|
|
|
20 |
- static
|
21 |
---
|
22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
# Codestral-22B-v0.1-IMat-GGUF
|
24 |
_Llama.cpp imatrix quantization of mistralai/Codestral-22B-v0.1 (bullerwins/Codestral-22B-v0.1-hf)_
|
25 |
|
26 |
Original model: [mistralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1)
|
27 |
+
Quantized HF Model: [legraphista/Codestral-22B-v0.1-hf-FIM-fix](https://huggingface.co/legraphista/Codestral-22B-v0.1-hf-FIM-fix)
|
28 |
Original dtype: `BF16` (`bfloat16`)
|
29 |
+
Quantized by: llama.cpp [b3046](https://github.com/ggerganov/llama.cpp/releases/tag/b3046)
|
30 |
IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
|
31 |
|
32 |
+
- [Files](#files)
|
33 |
+
- [IMatrix](#imatrix)
|
34 |
+
- [Common Quants](#common-quants)
|
35 |
+
- [All Quants](#all-quants)
|
36 |
+
- [Downloading using huggingface-cli](#downloading-using-huggingface-cli)
|
37 |
+
- [Inference](#inference)
|
38 |
+
- [Simple chat template](#simple-chat-template)
|
39 |
+
- [Chat template with system prompt](#chat-template-with-system-prompt)
|
40 |
+
- [FIM / Fill In the Middle](#fim-fill-in-the-middle)
|
41 |
+
- [Llama.cpp](#llama-cpp)
|
42 |
+
- [FAQ](#faq)
|
43 |
+
- [Why is the IMatrix not applied everywhere?](#why-is-the-imatrix-not-applied-everywhere)
|
44 |
+
- [How do I merge a split GGUF?](#how-do-i-merge-a-split-gguf)
|
45 |
|
46 |
---
|
47 |
|