Update README.md
Browse files
README.md
CHANGED
@@ -34,7 +34,6 @@ python fp8_cast_bf16.py --input-fp8-hf-path /home/admin/models/deepseek-ai/DeepS
|
|
34 |
```
|
35 |
## BF16 to f16.gguf
|
36 |
1. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) conversion program to convert DeepSeek-V3-bf16 to gguf format, requires an additional approximately 1.3 TB of space.
|
37 |
-
**Note:** this model requires [Ollama 0.5.5](https://github.com/ollama/ollama/releases/tag/v0.5.5)
|
38 |
```
|
39 |
python convert_hf_to_gguf.py /home/admin/models/deepseek-ai/DeepSeek-V3-bf16 --outfile /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gguf --outtype f16
|
40 |
```
|
@@ -47,4 +46,12 @@ llama-quantize /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gg
|
|
47 |
3. Use llama-cli to test.
|
48 |
```
|
49 |
llama-cli -m /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-Q2_K.gguf -n 2048
|
50 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
```
|
35 |
## BF16 to f16.gguf
|
36 |
1. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) conversion program to convert DeepSeek-V3-bf16 to gguf format, requires an additional approximately 1.3 TB of space.
|
|
|
37 |
```
|
38 |
python convert_hf_to_gguf.py /home/admin/models/deepseek-ai/DeepSeek-V3-bf16 --outfile /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gguf --outtype f16
|
39 |
```
|
|
|
46 |
3. Use llama-cli to test.
|
47 |
```
|
48 |
llama-cli -m /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-Q2_K.gguf -n 2048
|
49 |
+
```
|
50 |
+
|
51 |
+
## Use with ollama
|
52 |
+
**Note:** this model requires [Ollama 0.5.5](https://github.com/ollama/ollama/releases/tag/v0.5.5)
|
53 |
+
|
54 |
+
You can use [huihui_ai/deepseek-v3:671b-q2_K](https://ollama.com/huihui_ai/deepseek-v3:671b-q2_K) directly
|
55 |
+
```
|
56 |
+
ollama run huihui_ai/deepseek-v3:671b-q2_K
|
57 |
+
```
|