| | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate | | |
| | ------ | --------- | ------- | -------- | ------------ | -------------- | | |
| | IQ1_S | 693 | 80.4634 | 13.61 | 12.16 | 1.33 | | |
| | IQ1_M | 734 | 39.7732 | 14.41 | 24.60 | 0.61 | | |
| | IQ2_XXS | 803 | 20.3081 | 15.77 | 48.18 | 0.30 | | |
| | IQ2_XS | 864 | 15.7232 | 16.97 | 62.23 | 0.23 | | |
| | IQ2_S | 921 | 14.1473 | 18.09 | 69.16 | 0.21 | | |
| | IQ2_M | 976 | 12.5527 | 19.17 | 77.95 | 0.18 | | |
| | Q2_K_S | 985 | 13.7018 | 19.34 | 71.41 | 0.20 | | |
| | Q2_K | 1045 | 12.5399 | 20.52 | 78.03 | 0.19 | | |
| | IQ3_XXS | 1068 | 11.1884 | 20.97 | 87.45 | 0.16 | | |
| | IQ3_XS | 1152 | 10.8551 | 22.62 | 90.14 | 0.16 | | |
| | Q3_K_S | 1195 | 11.0653 | 23.47 | 88.43 | 0.16 | | |
| | IQ3_S | 1201 | 10.6916 | 23.59 | 91.51 | 0.15 | | |
| | IQ3_M | 1233 | 10.6124 | 24.21 | 92.20 | 0.15 | | |
| | Q3_K_M | 1298 | 10.3392 | 25.49 | 94.63 | 0.15 | | |
| | Q3_K_L | 1391 | 10.2274 | 27.32 | 95.67 | 0.15 | | |
| | IQ4_XS | 1435 | 10.0262 | 28.18 | 97.59 | 0.15 | | |
| | Q4_0 | 1503 | 10.1964 | 29.52 | 95.96 | 0.15 | | |
| | IQ4_NL | 1505 | 9.9962 | 29.56 | 97.88 | 0.15 | | |
| | Q4_K_S | 1507 | 10.0445 | 29.59 | 97.41 | 0.15 | | |
| | Q4_K_M | 1568 | 10.0122 | 30.79 | 97.72 | 0.15 | | |
| | Q4_1 | 1643 | 10.0464 | 32.27 | 97.39 | 0.15 | | |
| | Q5_K_S | 1786 | 9.8232 | 35.07 | 99.61 | 0.14 | | |
| | Q5_0 | 1789 | 9.8700 | 35.13 | 99.13 | 0.14 | | |
| | Q5_K_M | 1822 | 9.8565 | 35.78 | 99.27 | 0.14 | | |
| | Q5_1 | 1929 | 9.8203 | 37.88 | 99.64 | 0.14 | | |
| | Q6_K | 2091 | 9.8229 | 41.06 | 99.61 | 0.14 | | |
| | Q8_0 | 2707 | 9.7928 | 53.16 | 99.92 | 0.14 | | |
| | F16 | 5092 | 9.7845 | 100 | 100 | 0.14 | | |