Llama-3.1 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Sparse-Llama-3.1-2of4

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.1 Quantization

updated Nov 22

Neural Magic quantized Llama-3.1 models

neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8

Text Generation • Updated Oct 9 • 73k • 37
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8

Text Generation • Updated Oct 9 • 461k • 37
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w4a16

Text Generation • Updated Oct 10 • 126k • 28
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16

Text Generation • Updated 9 days ago • 101k • 23
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16

Text Generation • Updated Oct 23 • 3.68k • 9
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16

Text Generation • Updated Oct 10 • 2.74k • 12
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 261 • 14
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated 23 days ago • 372 • 2
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 838 • 5
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8

Text Generation • Updated Oct 10 • 12.4k • 16
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 1.54k • 5
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8

Text Generation • Updated Oct 9 • 3.68k • 31
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a8

Text Generation • Updated Oct 23 • 6.57k • 12
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 317 • 4
neuralmagic/Meta-Llama-3.1-8B-FP8

Text Generation • Updated Oct 9 • 11.7k • 5
neuralmagic/Meta-Llama-3.1-70B-FP8

Text Generation • Updated Oct 9 • 469 • 1
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a16

Text Generation • Updated Oct 9 • 329 • 1
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a8

Text Generation • Updated Oct 23 • 664 • 2
neuralmagic/Meta-Llama-3.1-405B-FP8

Text Generation • Updated Oct 9 • 40
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 315 • 2
neuralmagic/Meta-Llama-3.1-70B-quantized.w8a8

Text Generation • Updated Oct 9 • 36