perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	754	38.2733	12.80	25.21	0.68
IQ1_M	810	23.9542	13.75	40.28	0.41
IQ2_XXS	904	22.7872	15.34	42.34	0.41
IQ2_XS	983	14.4025	16.68	66.99	0.24
IQ2_S	1012	13.6118	17.17	70.88	0.22
IQ2_M	1087	12.6560	18.45	76.24	0.20
Q2_K_S	1142	12.4745	19.38	77.35	0.20
Q2_K	1215	11.5168	20.62	83.78	0.18
IQ3_XXS	1223	10.8327	20.76	89.07	0.17
IQ3_XS	1327	10.2693	22.52	93.96	0.16
Q3_K_S	1386	32.8121	23.52	29.41	0.62
IQ3_S	1389	10.2409	23.57	94.22	0.16
IQ3_M	1419	10.1950	24.08	94.64	0.16
Q3_K_M	1516	20.7076	25.73	46.59	0.36
Q3_K_L	1628	20.6384	27.63	46.75	0.36
IQ4_XS	1658	9.9013	28.14	97.45	0.15
IQ4_NL	1740	9.8926	29.53	97.53	0.15
Q4_0	1743	10.0757	29.58	95.76	0.16
Q4_K_S	1749	9.8497	29.68	97.96	0.15
Q4_K_M	1840	9.8160	31.23	98.29	0.15
Q4_1	1903	9.8176	32.30	98.28	0.15
Q5_K_S	2069	9.7456	35.12	99.00	0.15
Q5_0	2074	9.7237	35.20	99.23	0.15
Q5_K_M	2121	9.7331	36.00	99.13	0.15
Q5_1	2234	9.7260	37.92	99.20	0.15
Q6_K	2420	9.6939	41.07	99.53	0.15
Q8_0	3133	9.6710	53.17	99.77	0.15
F16	5892	9.6486	100	100	0.15