huihui-ai commited on
Commit
9e76422
·
verified ·
1 Parent(s): 70fe8b6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -34,7 +34,6 @@ python fp8_cast_bf16.py --input-fp8-hf-path /home/admin/models/deepseek-ai/DeepS
34
  ```
35
  ## BF16 to f16.gguf
36
  1. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) conversion program to convert DeepSeek-V3-bf16 to gguf format, requires an additional approximately 1.3 TB of space.
37
- **Note:** this model requires [Ollama 0.5.5](https://github.com/ollama/ollama/releases/tag/v0.5.5)
38
  ```
39
  python convert_hf_to_gguf.py /home/admin/models/deepseek-ai/DeepSeek-V3-bf16 --outfile /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gguf --outtype f16
40
  ```
@@ -47,4 +46,12 @@ llama-quantize /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gg
47
  3. Use llama-cli to test.
48
  ```
49
  llama-cli -m /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-Q2_K.gguf -n 2048
50
- ```
 
 
 
 
 
 
 
 
 
34
  ```
35
  ## BF16 to f16.gguf
36
  1. Use the [llama.cpp](https://github.com/ggerganov/llama.cpp) conversion program to convert DeepSeek-V3-bf16 to gguf format, requires an additional approximately 1.3 TB of space.
 
37
  ```
38
  python convert_hf_to_gguf.py /home/admin/models/deepseek-ai/DeepSeek-V3-bf16 --outfile /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-f16.gguf --outtype f16
39
  ```
 
46
  3. Use llama-cli to test.
47
  ```
48
  llama-cli -m /home/admin/models/deepseek-ai/DeepSeek-V3-bf16/ggml-model-Q2_K.gguf -n 2048
49
+ ```
50
+
51
+ ## Use with ollama
52
+ **Note:** this model requires [Ollama 0.5.5](https://github.com/ollama/ollama/releases/tag/v0.5.5)
53
+
54
+ You can use [huihui_ai/deepseek-v3:671b-q2_K](https://ollama.com/huihui_ai/deepseek-v3:671b-q2_K) directly
55
+ ```
56
+ ollama run huihui_ai/deepseek-v3:671b-q2_K
57
+ ```