Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Paper • 2405.03594 • Published May 6 • 7
"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization Paper • 2411.02355 • Published 5 days ago • 44
Llama-3.2 Quantization Collection Llama 3.2 models quantized by Neural Magic • 9 items • Updated Sep 26 • 9
Llama-3.1 Quantization Collection Neural Magic quantized Llama-3.1 models • 21 items • Updated Sep 26 • 38
INT8 LLMs for vLLM Collection Accurate INT8 quantized models by Neural Magic, ready for use with vLLM! • 50 items • Updated Sep 26 • 9
