The UTF-8-Everywhere Manifesto
tl:dr; Use UTF-8 when you need to use unicode with legacy APIs, never anywhere else.UNIX isn't UTF-8 because UTF-8 is better, UNIX is UTF-8 because you can pass UTF-8 strings to functions that expect ASCII and it kinda works. This is really the only thing you need to know about UTF-8 and why it's better.There are few pieces of software that don't have to talk to legacy APIs that store strings natively in UTF-8.C# and Java are probably the best examples of software that was engineered from the ground up and thus uses UTF-16 internally because it's much less likely to run into issues like String.length returning 32 yet only containing 31 characters. If you use UTF-8 expect this result anytime a string contains a real genuine apostrophe."UTF-8 and UTF-32 result the same order when sorted lexicographically. UTF-16 does not."This is complete and utter bullshit, to sort a string lexicographically you need to decode it, if you've decoded the string into UNICODE then they sort the exact same way.There are lots of gotchas for sorting UNICODE strings including normalization because you can write the semantically equivalent strings in unicode multiple ways. eg. ligatures.If you're sorting bit strings that happen to contain UTF-8/32 then you're not sorting lexicographically and your results will be screwed up anyway.