timonecomcastnet tim peters tim id prefer strip html tags everything last time tried still bad effects error rates corpora guido corpora biased respect though newsgroups strong social taboo posting html many peoples personal inboxes quite abundant violent agreement comments tokenizerpy say strongly possible ive repeated endlessly long one serious testing dubious idea make code maximally clumsy use clpy task getting good ham corpus may prove bigger hurdle though saved mail doesnt reflect receive since save throw away selectively much past yup system picks everything tokens grahams proposed delete ham delete spam keys would probably work well motivated geeks paul svensson pointed probably wouldnt work nearly well real people ah explains html tags didnt get stripped id offer add optional argument tokenize theyd get stripped gets glossed third time would feel much like loss ill bite sounds like good idea strip html case id like see improves fp rate corpus ill soon check change def tokenize_bodyself msg retain_pure_html_tagsfalse generate stream tokens email message multipartalternative section textplain texthtml sections texthtml section ignored may good idea eg sections may different content html tags always stripped textplain sections default html tags also stripped texthtml sections however hurts false negative rate tims complangpython tests htmlonly messages almost never legitimate traffic optional argument retain_pure_html_tags specified true html tags retained texthtml sections cvs establish new baseline first checked purewin change wee hours cut fp fn rates tests mon 09 sep 2002 104552 0400 spambayes test sets