text_combined: jeremyalummitedu jeremy hylton tp tim peters writes tp jeremy hylton total collections 1100 messages trained 11005 messages tp im reading trained 220 spam tp 220 ham thats less 10 sizes tp training sets ive using please try experiment train tp 550 test 550 helps lot results stock tokenizer training 644 hams 557 spams 0000 10413 1398 6104 1398 5027 training 644 hams 557 spams 0000 8259 1242 2873 1242 5745 training 644 hams 557 spams 1398 5206 1398 4488 0000 9336 training 644 hams 557 spams 1553 5206 1553 5027 0000 9874 total false pos 139 539596273292 total false neg 970 435368043088 results tokenizer looks headers except date received xfrom_ training 644 hams 557 spams 0000 7540 0932 4847 0932 3232 training 644 hams 557 spams 0000 7181 0621 2873 0621 4847 training 644 hams 557 spams 1087 4129 1087 3052 0000 6822 training 644 hams 557 spams 0776 3411 0776 3411 0000 6463 total false pos 97 376552795031 total false neg 738 331238779174 safe conclude avoiding cleverness headers good thing tp times making random split time wont tp long discover directories individual files tp lot easier work eg randomshuffle makes kind tp thing trivial havent looked mboxtestpy im using randomshuffle dont need many files make arbitrary selection messages mbox ill report results theyre done jeremy sat 7 sep 2002 131817 0400 spambayes understanding high false negative rate