Hugging Quants

AI & ML interests

Optimised quants for high-throughput deployments! Compatible with Transformers, TGI & vLLM 🤗

hugging-quants 's models 21

hugging-quants/Llama-4-Scout-17B-16E-Instruct-fbgemm

Any-to-Any • 109B • Updated Apr 9 • 8 • 2

hugging-quants/Llama-4-Scout-17B-16E-Instruct-fbgemm-unfused

Any-to-Any • 109B • Updated Apr 9 • 7 • 2

hugging-quants/gemma-2-9b-it-AWQ-INT4

Text Generation • 2B • Updated Oct 17, 2024 • 8.76k • 7

hugging-quants/Mixtral-8x7B-Instruct-v0.1-AWQ-INT4

Text Generation • 6B • Updated Oct 7, 2024 • 10.2k

hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF

Text Generation • 1B • Updated Sep 25, 2024 • 34.5k • 18

hugging-quants/Llama-3.2-1B-Instruct-Q8_0-GGUF

Text Generation • 1B • Updated Sep 25, 2024 • 448k • 35

hugging-quants/Llama-3.2-3B-Instruct-Q4_K_M-GGUF

Text Generation • 3B • Updated Sep 25, 2024 • 23k • 26

hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF

Text Generation • 3B • Updated Sep 25, 2024 • 981 • 52

hugging-quants/Meta-Llama-3.1-405B-BNB-NF4

Text Generation • 211B • Updated Sep 16, 2024 • 83 • 2

hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • 214B • Updated Sep 16, 2024 • 26 • 5

hugging-quants/Meta-Llama-3.1-405B-BNB-NF4-BF16

Text Generation • 111B • Updated Sep 16, 2024 • 56 • 2

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • 59B • Updated Sep 13, 2024 • 11.8k • 36

hugging-quants/Meta-Llama-3.1-8B-BNB-NF4

Text Generation • 5B • Updated Aug 8, 2024 • 60 • 1

hugging-quants/Meta-Llama-3.1-70B-BNB-NF4-BF16

Text Generation • 20B • Updated Aug 8, 2024 • 27 • 2

hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • 5B • Updated Aug 8, 2024 • 529 • 8

hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • 2B • Updated Aug 7, 2024 • 210k • 80

hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • 11B • Updated Aug 7, 2024 • 101k • 23

hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • 59B • Updated Aug 7, 2024 • 321 • 16

hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • 2B • Updated Aug 7, 2024 • 8.81k • 40

hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • 11B • Updated Aug 7, 2024 • 86.6k • 107

hugging-quants/Meta-Llama-3.1-8B-BNB-NF4-BF16

Text Generation • 3B • Updated Jul 28, 2024 • 9