timonecomcastnet tim peters anthony baxter well ive finally got around pulling sf code starting absolutely zero local modifications see following many runs summarizing many hamspam training set many prediction sets error rates run ratespy output file effect set sizes accuracy rates isnt known ive informally reported results controlled experiments jeremy reported improved accuracy doubling training set size wasnt controlled experiment things besides training set size changed ham distribution runs 589 items 000 35292 250 36 500 21 750 12 1000 6 9000 4 9250 8 9500 15 9750 441 spam distribution runs 504 items 000 393 250 17 500 18 750 12 1000 4 9000 11 9250 16 9500 45 9750 30226 next current task complete corpus ive got currently got 9000 ham 7800 spam 9200 currently unsorted im tossing using either hammie spamassassin initial sort previously ive used various forms grep keywords little gui thing pop message let say spamham thats getting tedious yup tagging data mondo tedious mistakes hurt expect hammie much better job already hand grepping sure stare false positives get spam cant make available en masse look finding interesting uglies one thing ive seen consider anecdotal skip tokens end _lot_ fps probabilities favoring ham spam skip token produced lieu word 12 chars long without highbit characters possible helped raw html produces lots however youre running current cvs tokenizerretain_pure_html_tags defaults false html decorations vanish body tokenization tue 10 sep 2002 150136 0400 spambayes current histograms