DeepSeek-R1-Llama-8B-F32-GGUF

DeepSeek-R1-Llama-8B-F32-GGUF is a quantized version of DeepSeek-R1-Distill-Llama-8B, trained using reinforcement learning (RL) directly on the base model without relying on supervised fine-tuning (SFT) as a preliminary step. This approach enables the model to explore chain-of-thought (CoT) reasoning for solving complex problems, leading to the development of DeepSeek-R1-Zero. DeepSeek-R1-Zero demonstrates capabilities such as self-verification, reflection, and generating extended chains of thought

Model File

File Name	Size	Format	Notes
DeepSeek-R1-Llama-8B.BF16.gguf	15.6 GB	GGUF	BF16 precision model
DeepSeek-R1-Llama-8B.F16.gguf	16.1 GB	GGUF	FP16 precision model
DeepSeek-R1-Llama-8B.F32.gguf	32.1 GB	GGUF	FP32 precision model
.gitattributes	1.75 kB	Text	Git LFS tracking config
config.json	31 B	JSON	Model configuration file
README.md	767 B	Markdown	This readme file

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):