Commit
·
abb4a54
1
Parent(s):
d70433a
update readme
Browse files
README.md
CHANGED
@@ -228,29 +228,15 @@ model-index:
|
|
228 |
# ibm-granite/granite-20b-code-base-Q4_K_M-GGUF
|
229 |
This model was converted to GGUF format from [`ibm-granite/granite-20b-code-base`](https://huggingface.co/ibm-granite/granite-20b-code-base).
|
230 |
Refer to the [original model card](https://huggingface.co/ibm-granite/granite-20b-code-base) for more details on the model.
|
231 |
-
## Use with llama.cpp
|
232 |
-
|
233 |
-
Install llama.cpp through brew.
|
234 |
-
|
235 |
-
```bash
|
236 |
-
brew install ggerganov/ggerganov/llama.cpp
|
237 |
-
```
|
238 |
-
Invoke the llama.cpp server or the CLI.
|
239 |
-
|
240 |
-
CLI:
|
241 |
|
242 |
-
|
243 |
-
|
244 |
-
|
245 |
-
|
246 |
-
Server:
|
247 |
-
|
248 |
-
```bash
|
249 |
-
llama-server --hf-repo ibm-granite/granite-20b-code-base-Q4_K_M-GGUF --model granite-20b-code-base.Q4_K_M.gguf -c 2048
|
250 |
-
```
|
251 |
|
252 |
-
|
|
|
253 |
|
254 |
-
|
255 |
-
|
256 |
```
|
|
|
228 |
# ibm-granite/granite-20b-code-base-Q4_K_M-GGUF
|
229 |
This model was converted to GGUF format from [`ibm-granite/granite-20b-code-base`](https://huggingface.co/ibm-granite/granite-20b-code-base).
|
230 |
Refer to the [original model card](https://huggingface.co/ibm-granite/granite-20b-code-base) for more details on the model.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
231 |
|
232 |
+
## Use with llama.cpp
|
233 |
+
```shell
|
234 |
+
git clone https://github.com/ggerganov/llama.cpp
|
235 |
+
cd llama.cpp
|
|
|
|
|
|
|
|
|
|
|
236 |
|
237 |
+
# install
|
238 |
+
make
|
239 |
|
240 |
+
# run generation
|
241 |
+
./main -m granite-20b-code-base-Q4_K_M-GGUF/granite-20b-code-base.Q4_K_M.gguf -n 128 -p "def generate_random(x: int):" --color
|
242 |
```
|