| # GGUF Quantized Models | |
| This repository contains GGUF quantized versions of language models for efficient inference. | |
| ## Files | |
| The following GGUF files are available in this repository: | |
| - `DeepHermes-3-Llama-3-3B-Preview-q4.gguf` (1.8G) | |
| - `DeepHermes-3-Llama-3-3B-Preview-q5.gguf` (2.2G) | |
| - `DeepHermes-3-Llama-3-3B-Preview-q6.gguf` (2.5G) | |
| - `DeepHermes-3-Llama-3-3B-Preview-q8.gguf` (3.2G) | |
| ## Usage | |
| These models are compatible with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar frameworks. | |
| Example usage with llama.cpp: | |
| ```bash | |
| ./main -m /path/to/model.gguf -p "Hello, I am a language model" -n 128 | |
| ``` | |
| ## Upload Information | |
| Files were uploaded on Tue Mar 11 04:28:49 PDT 2025 | |