Please quantize a Q5_K_S version

#9
by fakezeta - opened

The performance gain going Q5 usually is worth the extra bpw from Q4_K_L: could you kindly quantize with your UD 2.0?
Or the provided Q4_K_XL should have the same performance of Q5_K_S?

Thank you

Unsloth AI org

Thanks for the suggestion we'll see what we can do :)

Q4 XL should be very similar to Q5 yes

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment