timonecomcastnet tim peters tim id prefer strip html tags everything last time tried still bad effects error rates corpora full test results without html tag stripping included html comment block comment block also says xxx another way found slash fn rate decision xxx strip html htmlonly msgs revisited weve since done several things gave significant fn rate reductions test alas stripping html tags text still hurts fn rate test data false positive percentages 0000 0000 tied 0000 0000 tied 0050 0075 lost 5000 0025 0025 tied 0075 0025 6667 0000 0000 tied 0100 0100 tied 0050 0075 lost 5000 0025 0025 tied 0025 0000 10000 0050 0075 lost 5000 0050 0050 tied 0050 0025 5000 0000 0000 tied 0000 0000 tied 0075 0075 tied 0025 0025 tied 0000 0000 tied 0025 0025 tied 0050 0050 tied 3 times tied 14 times lost 3 times total unique fp went 13 11 false negative percentages 0327 0400 lost 2232 0400 0400 tied 0327 0473 lost 4465 0691 0654 535 0545 0473 1321 0291 0364 lost 2509 0218 0291 lost 3349 0654 0654 tied 0364 0473 lost 2995 0291 0327 lost 1237 0327 0291 1101 0691 0654 535 0582 0655 lost 1254 0291 0400 lost 3746 0364 0436 lost 1978 0436 0582 lost 3349 0436 0364 1651 0218 0291 lost 3349 0291 0400 lost 3746 0254 0327 lost 2874 5 times tied 2 times lost 13 times total unique fn went 106 122 last time tried see tokenizerpy comments fn rate stripping tags ranged 0982 1781 median 134 weve made tons progress fn rate since mere presence html tags still remains significant clue clpy traffic im left comment xxx another way found slash fn rate decision xxx strip html htmlonly msgs revisited want take focus away clpy traffic cant say effect html stripping would dont suitable test data measure sun 08 sep 2002 034828 0400 spambayes test sets