greg freemyer tlvognmhkkorgmailcom feb 19 2008 801 petr cerny wrote greg freemyer wrote huge text file 17 million lines full unicode ascii text half half note disk space copies problem need manipulate file also 30 line file full ascii text need search large file occurrences keywords 30 line file ignoring unicode issue could use grep fgrep egrep appropriate args idea handle unicode issue ascii subset utf8 unicode coding utf8 problem case wonder manage create file interspersed single fixed size multibyte encodings guess would split file somehow best regards petr see im bit confused worked much unicode linux unicode file looks like normal ascii null every byte think thats normal limited exposure unicode status im office recreated 17 million line file last night pure unicode cant test couple hours claim use grep search unicode file ascii text pattern yesterday looked grep switch would tell file searched unicode didnt see one guess could automatic actually attempt feasible tool convert unicode file ascii file would great text interested convert ascii obvious way need mappings basically need get rid nulls greg greg freemyer litigation triage solutions specialist httpwwwlinkedincomingregfreemyer first 99 days litigation white paper httpwwwnorcrossgroupcomformswhitepapers9920days20whitepaperpdf norcross group intersection evidence technology httpwwwnorcrossgroupcom thu 07 aug 2008 001605 0500 opensuse unicode v ascii