# GGUF Quantized Models

This repository contains GGUF quantized versions of language models for efficient inference.

## Files

The following GGUF files are available in this repository:

- `DeepHermes-3-Llama-3-3B-Preview-q4.gguf` (1.8G)
- `DeepHermes-3-Llama-3-3B-Preview-q5.gguf` (2.2G)
- `DeepHermes-3-Llama-3-3B-Preview-q6.gguf` (2.5G)
- `DeepHermes-3-Llama-3-3B-Preview-q8.gguf` (3.2G)

## Usage

These models are compatible with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar frameworks.

Example usage with llama.cpp:
```bash
./main -m /path/to/model.gguf -p "Hello, I am a language model" -n 128
```

## Upload Information
Files were uploaded on Tue Mar 11 04:28:49 PDT 2025