Update README.md
Browse files
README.md
CHANGED
@@ -7,23 +7,25 @@ language:
|
|
7 |
|
8 |
---
|
9 |
|
10 |
-
# Model
|
11 |
|
12 |
|
13 |
-
Here is a Quantized version of Llama-3.1-70B-Instruct using GGUF
|
14 |
|
15 |
-
GGUF is designed for use with GGML and other executors
|
16 |
-
GGUF was developed by @ggerganov who is also the developer of llama.cpp, a popular C/C++ LLM inference framework
|
17 |
-
Models initially developed in frameworks like PyTorch can be converted to GGUF format for use with those engines
|
18 |
|
19 |
|
20 |
-
## Uploaded Quantization Types
|
21 |
|
22 |
-
Currently, I have uploaded 2 quantized versions
|
23 |
|
24 |
-
Q5_K_M : - large, very low quality loss
|
25 |
-
|
26 |
-
|
|
|
|
|
27 |
|
28 |
### All Quantization Types Possible
|
29 |
|
|
|
7 |
|
8 |
---
|
9 |
|
10 |
+
# Model<br>
|
11 |
|
12 |
|
13 |
+
Here is a Quantized version of Llama-3.1-70B-Instruct using GGUF<br>
|
14 |
|
15 |
+
GGUF is designed for use with GGML and other executors.<br>
|
16 |
+
GGUF was developed by @ggerganov who is also the developer of llama.cpp, a popular C/C++ LLM inference framework.<br>
|
17 |
+
Models initially developed in frameworks like PyTorch can be converted to GGUF format for use with those engines.<br>
|
18 |
|
19 |
|
20 |
+
## Uploaded Quantization Types<br>
|
21 |
|
22 |
+
Currently, I have uploaded 2 quantized versions:<br>
|
23 |
|
24 |
+
Q5_K_M : - large, very low quality loss<br>
|
25 |
+
|
26 |
+
and<br>
|
27 |
+
|
28 |
+
Q8_0 : - very large, extremely low quality loss<br>
|
29 |
|
30 |
### All Quantization Types Possible
|
31 |
|