Check out AutoRound, SOTA LLM quantization algorithm across 2-4 bits without adding any inference overhead to any model
text
Check out AutoRound, SOTA LLM quantization algorithm across 2-4 bits without adding any inference overhead to any model
new_line
paper: 
text
paper: 
https://arxiv.org/abs/2309.05516
https://arxiv.org/abs/2309.05516
link
new_line
github: 
text
github: 
https://github.com/intel/auto-round
https://github.com/intel/auto-round
link
new_line
lowbits leaderboard: 
text
lowbits leaderboard: 
https://huggingface.co/spaces/Intel/low-bit-leaderboard
https://huggingface.co/spaces/Intel/low-bit-leaderboard
link
new_line
Check out AutoRound, SOTA LLM quantization algorithm across 2-4 bits without adding any inference overhead to any model
paper: https://arxiv.org/abs/2309.05516
github: https://github.com/intel/auto-round
lowbits leaderboard: https://huggingface.co/spaces/Intel/low-bit-leaderboard
