perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	693	80.4634	13.61	12.16	1.33
IQ1_M	734	39.7732	14.41	24.60	0.61
IQ2_XXS	803	20.3081	15.77	48.18	0.30
IQ2_XS	864	15.7232	16.97	62.23	0.23
IQ2_S	921	14.1473	18.09	69.16	0.21
IQ2_M	976	12.5527	19.17	77.95	0.18
Q2_K_S	985	13.7018	19.34	71.41	0.20
Q2_K	1045	12.5399	20.52	78.03	0.19
IQ3_XXS	1068	11.1884	20.97	87.45	0.16
IQ3_XS	1152	10.8551	22.62	90.14	0.16
Q3_K_S	1195	11.0653	23.47	88.43	0.16
IQ3_S	1201	10.6916	23.59	91.51	0.15
IQ3_M	1233	10.6124	24.21	92.20	0.15
Q3_K_M	1298	10.3392	25.49	94.63	0.15
Q3_K_L	1391	10.2274	27.32	95.67	0.15
IQ4_XS	1435	10.0262	28.18	97.59	0.15
Q4_0	1503	10.1964	29.52	95.96	0.15
IQ4_NL	1505	9.9962	29.56	97.88	0.15
Q4_K_S	1507	10.0445	29.59	97.41	0.15
Q4_K_M	1568	10.0122	30.79	97.72	0.15
Q4_1	1643	10.0464	32.27	97.39	0.15
Q5_K_S	1786	9.8232	35.07	99.61	0.14
Q5_0	1789	9.8700	35.13	99.13	0.14
Q5_K_M	1822	9.8565	35.78	99.27	0.14
Q5_1	1929	9.8203	37.88	99.64	0.14
Q6_K	2091	9.8229	41.06	99.61	0.14
Q8_0	2707	9.7928	53.16	99.92	0.14
F16	5092	9.7845	100	100	0.14