timonecomcastnet tim peters neil trained classifier using 3 sets 500 ham spam missing half test run results due cmppy bug since fixed custom fiddling figures 3 reported runs false positive percentages 0187 0749 0780 total unique fp 19 false negative percentages 2072 2448 0574 total unique fn 43 total unique figures counts 6 runs individualrun fp fn percentages missing 3 runs jeremy reported custom fiddling figures 4 sets 600 ham spam false positive percentages 0000 1398 1398 0000 1242 1242 1398 1398 0000 1553 1553 0000 total unique fp 139 false negative percentages 10413 6104 5027 8259 2873 5745 5206 4488 9336 5206 5027 9874 total unique fn 970 things clearly working much better neil reported significant improvements fn fp rates folding header lines neal added received analysis base tokenizers header analysis jeremy skipped base tokenizers header analysis completely added basesubjectlinelike casefolded tokenization almost header lines excepting received data xfrom_ suspect starting xvm try 5 random pairs 500ham 500spam subsets test data see false positive percentages 0000 0000 0200 0000 0200 0000 0200 0000 0000 0200 0400 0000 0200 0000 0200 0400 0000 0400 0200 0600 total unique fp 10 false negative percentages 0800 0400 0200 0600 1000 0000 0600 1200 1200 0800 0400 0800 1800 0800 0400 1000 1000 0400 0000 0600 total unique fn 36 much closer neil saw still looks better another run disjoint 5 random pairs looked much total unique fp rose 12 fn fell 27 third run another set disjoint 5 random pairs likewise fp 12 fn 40 im pretty confident going matter random subsets 500 take data hard conclude anything given jeremys much worse results line neils results id suspect ive overtuned algorithm statistical quirks corpora sun 08 sep 2002 154813 0400 spambayes testing results