mayank-mishra commited on
Commit
abb4a54
·
1 Parent(s): d70433a

update readme

Browse files
Files changed (1) hide show
  1. README.md +8 -22
README.md CHANGED
@@ -228,29 +228,15 @@ model-index:
228
  # ibm-granite/granite-20b-code-base-Q4_K_M-GGUF
229
  This model was converted to GGUF format from [`ibm-granite/granite-20b-code-base`](https://huggingface.co/ibm-granite/granite-20b-code-base).
230
  Refer to the [original model card](https://huggingface.co/ibm-granite/granite-20b-code-base) for more details on the model.
231
- ## Use with llama.cpp
232
-
233
- Install llama.cpp through brew.
234
-
235
- ```bash
236
- brew install ggerganov/ggerganov/llama.cpp
237
- ```
238
- Invoke the llama.cpp server or the CLI.
239
-
240
- CLI:
241
 
242
- ```bash
243
- llama-cli --hf-repo ibm-granite/granite-20b-code-base-Q4_K_M-GGUF --model granite-20b-code-base.Q4_K_M.gguf -p "def generate(random_seed: int):"
244
- ```
245
-
246
- Server:
247
-
248
- ```bash
249
- llama-server --hf-repo ibm-granite/granite-20b-code-base-Q4_K_M-GGUF --model granite-20b-code-base.Q4_K_M.gguf -c 2048
250
- ```
251
 
252
- Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
 
253
 
254
- ```
255
- git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m granite-20b-code-base.Q4_K_M.gguf -n 128
256
  ```
 
228
  # ibm-granite/granite-20b-code-base-Q4_K_M-GGUF
229
  This model was converted to GGUF format from [`ibm-granite/granite-20b-code-base`](https://huggingface.co/ibm-granite/granite-20b-code-base).
230
  Refer to the [original model card](https://huggingface.co/ibm-granite/granite-20b-code-base) for more details on the model.
 
 
 
 
 
 
 
 
 
 
231
 
232
+ ## Use with llama.cpp
233
+ ```shell
234
+ git clone https://github.com/ggerganov/llama.cpp
235
+ cd llama.cpp
 
 
 
 
 
236
 
237
+ # install
238
+ make
239
 
240
+ # run generation
241
+ ./main -m granite-20b-code-base-Q4_K_M-GGUF/granite-20b-code-base.Q4_K_M.gguf -n 128 -p "def generate_random(x: int):" --color
242
  ```