File size: 2,274 Bytes
5e81a72 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------ | --------- | ------- | -------- | ------------ | -------------- |
| IQ1_S | 693 | 80.4634 | 13.61 | 12.16 | 1.33 |
| IQ1_M | 734 | 39.7732 | 14.41 | 24.60 | 0.61 |
| IQ2_XXS | 803 | 20.3081 | 15.77 | 48.18 | 0.30 |
| IQ2_XS | 864 | 15.7232 | 16.97 | 62.23 | 0.23 |
| IQ2_S | 921 | 14.1473 | 18.09 | 69.16 | 0.21 |
| IQ2_M | 976 | 12.5527 | 19.17 | 77.95 | 0.18 |
| Q2_K_S | 985 | 13.7018 | 19.34 | 71.41 | 0.20 |
| Q2_K | 1045 | 12.5399 | 20.52 | 78.03 | 0.19 |
| IQ3_XXS | 1068 | 11.1884 | 20.97 | 87.45 | 0.16 |
| IQ3_XS | 1152 | 10.8551 | 22.62 | 90.14 | 0.16 |
| Q3_K_S | 1195 | 11.0653 | 23.47 | 88.43 | 0.16 |
| IQ3_S | 1201 | 10.6916 | 23.59 | 91.51 | 0.15 |
| IQ3_M | 1233 | 10.6124 | 24.21 | 92.20 | 0.15 |
| Q3_K_M | 1298 | 10.3392 | 25.49 | 94.63 | 0.15 |
| Q3_K_L | 1391 | 10.2274 | 27.32 | 95.67 | 0.15 |
| IQ4_XS | 1435 | 10.0262 | 28.18 | 97.59 | 0.15 |
| Q4_0 | 1503 | 10.1964 | 29.52 | 95.96 | 0.15 |
| IQ4_NL | 1505 | 9.9962 | 29.56 | 97.88 | 0.15 |
| Q4_K_S | 1507 | 10.0445 | 29.59 | 97.41 | 0.15 |
| Q4_K_M | 1568 | 10.0122 | 30.79 | 97.72 | 0.15 |
| Q4_1 | 1643 | 10.0464 | 32.27 | 97.39 | 0.15 |
| Q5_K_S | 1786 | 9.8232 | 35.07 | 99.61 | 0.14 |
| Q5_0 | 1789 | 9.8700 | 35.13 | 99.13 | 0.14 |
| Q5_K_M | 1822 | 9.8565 | 35.78 | 99.27 | 0.14 |
| Q5_1 | 1929 | 9.8203 | 37.88 | 99.64 | 0.14 |
| Q6_K | 2091 | 9.8229 | 41.06 | 99.61 | 0.14 |
| Q8_0 | 2707 | 9.7928 | 53.16 | 99.92 | 0.14 |
| F16 | 5092 | 9.7845 | 100 | 100 | 0.14 |
|