InferenceIllusionist
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1 |
---
|
2 |
-
base_model: mistralai/
|
3 |
library_name: transformers
|
4 |
quantized_by: InferenceIllusionist
|
5 |
language:
|
@@ -16,7 +16,6 @@ tags:
|
|
16 |
- iMat
|
17 |
- gguf
|
18 |
- Mistral
|
19 |
-
- Math
|
20 |
license: apache-2.0
|
21 |
---
|
22 |
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
|
@@ -25,11 +24,11 @@ license: apache-2.0
|
|
25 |
|
26 |
<b>Important Note: Inferencing is *only* available on this fork of llama.cpp at the moment: https://github.com/ggerganov/llama.cpp/pull/8604
|
27 |
|
28 |
-
Other front-ends like the main branch of llama.cpp, kobold.cpp, and text-generation-web-ui may not work as intended
|
29 |
|
30 |
Quantized from fp16.
|
31 |
* Weighted quantizations were creating using fp16 GGUF and groups_merged.txt in 92 chunks and n_ctx=512
|
32 |
-
* Static fp16 also included in repo
|
33 |
|
34 |
For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
|
35 |
|
|
|
1 |
---
|
2 |
+
base_model: mistralai/Mistral-Nemo-Instruct-2407
|
3 |
library_name: transformers
|
4 |
quantized_by: InferenceIllusionist
|
5 |
language:
|
|
|
16 |
- iMat
|
17 |
- gguf
|
18 |
- Mistral
|
|
|
19 |
license: apache-2.0
|
20 |
---
|
21 |
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
|
|
|
24 |
|
25 |
<b>Important Note: Inferencing is *only* available on this fork of llama.cpp at the moment: https://github.com/ggerganov/llama.cpp/pull/8604
|
26 |
|
27 |
+
Other front-ends like the main branch of llama.cpp, kobold.cpp, and text-generation-web-ui may not work as intended</b>
|
28 |
|
29 |
Quantized from fp16.
|
30 |
* Weighted quantizations were creating using fp16 GGUF and groups_merged.txt in 92 chunks and n_ctx=512
|
31 |
+
* Static fp16 will also be included in repo
|
32 |
|
33 |
For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
|
34 |
|