File size: 702 Bytes
cf716d4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# GGUF Quantized Models
This repository contains GGUF quantized versions of language models for efficient inference.
## Files
The following GGUF files are available in this repository:
- `DeepHermes-3-Llama-3-3B-Preview-q4.gguf` (1.8G)
- `DeepHermes-3-Llama-3-3B-Preview-q5.gguf` (2.2G)
- `DeepHermes-3-Llama-3-3B-Preview-q6.gguf` (2.5G)
- `DeepHermes-3-Llama-3-3B-Preview-q8.gguf` (3.2G)
## Usage
These models are compatible with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar frameworks.
Example usage with llama.cpp:
```bash
./main -m /path/to/model.gguf -p "Hello, I am a language model" -n 128
```
## Upload Information
Files were uploaded on Tue Mar 11 04:28:49 PDT 2025
|