Check out quantized weights from ISTA-DAS Lab directly in their organisation page: 
text
Check out quantized weights from ISTA-DAS Lab directly in their organisation page: 
https://cdn-avatars.huggingface.co/v1/production/uploads/628e0ce4e53bbd334577fcb0/TRPtgtSavYjDJOK3S1I8M.png
https://huggingface.co/ISTA-DASLab
ISTA-DASLab
org
resource
https://huggingface.co/ISTA-DASLab
 ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)
text
 ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)
new_line
new_line
Read more about these techniques below:
text
Read more about these techniques below:
new_line
new_line
AQLM paper: 
text
AQLM paper: 
Extreme Compression of Large Language Models via Additive Quantization (2401.06118)
https://hf.co/papers/2401.06118
2401.06118
paper
resource
https://hf.co/papers/2401.06118
new_line
QMoE: 
text
QMoE: 
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2310.16795)
https://huggingface.co/papers/2310.16795
2310.16795
paper
resource
https://huggingface.co/papers/2310.16795
new_line
new_line
Some useful links below:
text
Some useful links below:
new_line
new_line
AQLM repo: 
text
AQLM repo: 
https://github.com/Vahe1994/AQLM
https://github.com/Vahe1994/AQLM
link
new_line
How to use AQLM & transformers: 
text
How to use AQLM & transformers: 
https://huggingface.co/docs/transformers/quantization#aqlm
https://huggingface.co/docs/transformers/quantization#aqlm
link
new_line
How to use AQLM & PEFT: 
text
How to use AQLM & PEFT: 
https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
link
new_line
new_line
Great work from 
text
Great work from 
@BlackSamorez
mention
BlackSamorez
 and team !
text
 and team !
Check out quantized weights from ISTA-DAS Lab directly in their organisation page: https://huggingface.co/ISTA-DASLab ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)

Read more about these techniques below:

AQLM paper: https://hf.co/papers/2401.06118
QMoE: https://huggingface.co/papers/2310.16795

Some useful links below:

AQLM repo: https://github.com/Vahe1994/AQLM
How to use AQLM & transformers: https://huggingface.co/docs/transformers/quantization#aqlm
How to use AQLM & PEFT: https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion

Great work from @BlackSamorez and team !
❤️
alielfilali01 marcsun13 osanseviero samusenps FM-1976 mayank-mishra clem goodgame dillfrescott
👍
goodgame dillfrescott