Why even publish static quants in absence of i1 ?
#1
by
Alek01
- opened
Please at least provide i1 Q4_K_M and Q5_K_S to (barely) fit into 16Gb Vram!
The model has broken weights, so only static quants were made. But to answer your question, we make static quants because they are faster to create, and training-data independent. A static Q5_K_S is usually pretty indistinguishable to an imatrix Q5_K_S anyway.