<i>But, any ideas where the corpus for telling apart encodings should actually come from?</i><p>If you want a lot of fairly natural Japanese, try Wikipedia.  If you want something a bit more comprehensive and less restricted, the &quot;Balanced Corpus of Contemporary Written Japanese&quot; is a thing that exists, but you might have to jump through a few hoops.<p>That project is awesome, by the way.