Q3 30b A3b Pentiment - GPTQ - a allura-quants Collection

allura-quants 's Collections

Q3 8b Kintsugi - MLX

Q3 30b A3b Designant - EXL3

Q3 30b A3b Pentiment - GPTQ

Q3 30b A3b Pentiment - EXL2

Q3 30b A3b Pentiment - GPTQ

updated May 23

GPTQ quants made with both GPTQModel and Autoround (PSA, may not work ootb in vLLM --- working on it)

allura-quants/Q3-30b-A3b-Pentiment-W4G128-GPTQ-gptqmodel

5B • Updated May 22 • 1
allura-quants/Q3-30b-A3b-Pentiment-W8G128-GPTQ-gptqmodel

8B • Updated May 22 • 1
allura-quants/Q3-30b-A3b-Pentiment-W8G128-GPTQ-ar

8B • Updated May 22 • 1

Note not 100% sure on this one