Possible to do IQ2_M and IQ3_S/IQ3_M?
#6
by
Panchovix
- opened
Hi there, many thanks for your work!
It is possible to do these quants?
For example on Qwen 3 30B-A3B
There is a possiblity that IQ2_M would perform equally as Q2_K_XL for same or less size, and IQ3_S/IQ3_M compared to Q3_K_M or Q3_K_XL.
Benchmarks are always subjective and inconclusive unless done in super controlled environments. It's unlikely that a smaller model would perform much better than larger quants and might just be noise.