GGUF Quantized Models
This repository contains GGUF quantized versions of language models for efficient inference.
Files
The following GGUF files are available in this repository:
DeepHermes-3-Llama-3-3B-Preview-q4.gguf(1.8G)DeepHermes-3-Llama-3-3B-Preview-q5.gguf(2.2G)DeepHermes-3-Llama-3-3B-Preview-q6.gguf(2.5G)DeepHermes-3-Llama-3-3B-Preview-q8.gguf(3.2G)
Usage
These models are compatible with llama.cpp and similar frameworks.
Example usage with llama.cpp:
./main -m /path/to/model.gguf -p "Hello, I am a language model" -n 128
Upload Information
Files were uploaded on Tue Mar 11 04:28:49 PDT 2025