martin v löwis qpnyslvloewisde certainly agree target set documents small enough possible handcode encoding many applications however need examine content arbitrary least nonsmall set web documents name applications web search engines translation software ill question whether many programs web search engines translation software many challenges master fairly specialcased would expect need find answer character set detection anyway see bill janssens answer machine translation also documentbookmark management systems kinds document analysis market research seo etc sure specifically mind however expect also challenges example would expect msword documents frequent dont need character set detection word unicode need api look structure doc files substantiate documents feeds covers lot webi trying make point web whenever encoding specified specified incorrectly significant chunk exemplars firmly believe assumption false encoding comes software often correct time incorrect content editor type indeed one site pretty easy work around main use python processing analyzing hundreds millions web documents pretty easy see applications listed ok advantage would somebody working similar project gain chardet part standard library chardet algorithm also think good arguments excluding encoding detection maintenance reasons every case algorithm guessing wrong bug needs fixed stdlib unbounded commitment indeed thats meant initial remark people expect works correctly consequence unknowingly proceeding incorrect response complaining find produce incorrect answer chardet specifically usual standardlibrary remark applies cant become part standard library unless original author contributes anyway would hope group people would volunteer maintain threat removing stdlib volunteers go away many problems show regards martin _______________________________________________ pythondev mailing list zvlllneumpythonorg httpmailpythonorgmailmanlistinfopythondev unsubscribe httpmailpythonorgmailmanoptionspythondevpythondev40tangomucom fri 08 aug 2008 013834 0200 pythondev encoding detection standard library