How Python does Unicode
UCS-4 is essentially never the right choice.  It wastes space and thus messes up your cache.  UCS-2 can be the right choice if the language you&#x27;re encoding uses a lot of non-Latin glyphs (i.e. East Asian languages) but suffers from the same problem as UCS-2.  UTF-8 is a good default: for most strings it&#x27;s very compact, and for strings with a lot of multibyte codepoints it doesn&#x27;t compare too unfavorably with UTF-16.<p>Python 3 tried to have its cake and eat it too by choosing the most compact encoding depending on the string, but in practice this wastes a lot of space.  You&#x27;ll double (or heaven forbid <i></i>quadruple<i></i>) your string size because of a single codepoint, and these codepoints are almost always a small percentage of the string.  That&#x27;s actually why UTF-16 and UTF-8 exist.<p>It would have been better for strings to default to UTF-8, and to add an optional encoding so the programmer can specify what kind of encoding to use.  As it is now, in order to use (for example) UTF-16 strings in Python you have to keep them around as bytes, decode them to a string, perform string operations, and reencode them to bytes again.  Any benefit you get from using UTF-16 vanishes the moment you need to operate on it like a string, in other words.<p>I get that the idea was to maintain indexing via codepoint, but (again) in practice that&#x27;s not great: usually you want to index via grapheme -- if you want to index at all.<p>A better solution is to allow programmers to specify string encoding and default it to UTF-8.  From that, there&#x27;s a clear path to everything you&#x27;d want to do.