Llama-3.1 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.1 Quantization

updated Sep 26

Neural Magic quantized Llama-3.1 models

neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8

Text Generation • Updated Oct 9 • 51.6k • 32
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w4a16

Text Generation • Updated about 1 month ago • 19.4k • 25
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8

Text Generation • Updated Oct 9 • 131k • 32
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16

Text Generation • Updated 17 days ago • 14.3k • 21
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16

Text Generation • Updated 17 days ago • 5.5k • 8
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16

Text Generation • Updated about 1 month ago • 3.69k • 12
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic

Text Generation • Updated 22 days ago • 394 • 14
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated about 1 month ago • 896 • 2
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8-dynamic

Text Generation • Updated 22 days ago • 1.99k • 4
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8

Text Generation • Updated about 1 month ago • 12.7k • 11
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8-dynamic

Text Generation • Updated 22 days ago • 3.07k • 5
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8

Text Generation • Updated Oct 9 • 2.01k • 31
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a8

Text Generation • Updated 17 days ago • 9.41k • 11
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 1.41k • 3
neuralmagic/Meta-Llama-3.1-8B-FP8

Text Generation • Updated Oct 9 • 3.47k • 5
neuralmagic/Meta-Llama-3.1-70B-FP8

Text Generation • Updated Oct 9 • 611
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a16

Text Generation • Updated Oct 9 • 343 • 1
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a8

Text Generation • Updated 17 days ago • 559 • 1
neuralmagic/Meta-Llama-3.1-405B-FP8

Text Generation • Updated Oct 9 • 42
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 492 • 2