timonecomcastnet tim peters skip montanaro get mail several different email addresses frequently get duplicates triplicates moreplicates various spam messages saving spam later analysis havent always careful avoid saving duplicates wrote script time ago try minimize duplicates see calculating loose checksum still duplicates delete duplicates training people cant stop thinking classifier work best trained wholly random spattering real life real life contains duplicates thats classifier see would people interested script id happy extricate local modules check cvs sure think relevant maybe another purpose paul svensson thinking harder real people rest us may able get use approaches identify closely related spam example amount spam going end ham training data real life use sort similarity score piece known spam may aid finding purging mon 09 sep 2002 124655 0400 spambayes deleting duplicate spam training good idea