second-state
/

Llama-3.2-3B-Instruct-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

apepkuss79 commited on about 10 hours ago

Commit

1252cad

·

verified ·

1 Parent(s): bf944b3

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -37,7 +37,7 @@ tags:
 ## Run with LlamaEdge
-- LlamaEdge version: [v0.14.5](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.14.5) and above
 - Prompt template
@@ -126,4 +126,4 @@ tags:
 | [Llama-3.2-3B-Instruct-Q8_0.gguf](https://huggingface.co/second-state/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-Q8_0.gguf)     | Q8_0   | 8 | 1.32 GB| very large, extremely low quality loss - not recommended |
 | [Llama-3.2-3B-Instruct-f16.gguf](https://huggingface.co/second-state/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-f16.gguf)       | f16    | 16 | 2.48 GB| |
-*Quantized with llama.cpp b3807*

 ## Run with LlamaEdge
+- LlamaEdge version: [v0.16.5](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.16.5) and above
 - Prompt template
 | [Llama-3.2-3B-Instruct-Q8_0.gguf](https://huggingface.co/second-state/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-Q8_0.gguf)     | Q8_0   | 8 | 1.32 GB| very large, extremely low quality loss - not recommended |
 | [Llama-3.2-3B-Instruct-f16.gguf](https://huggingface.co/second-state/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-f16.gguf)       | f16    | 16 | 2.48 GB| |
+*Quantized with llama.cpp b4466*