Let's stop ascribing meaning to code points (2017)
As someone who&#x27;s written a lot of unicode-aware code in multiple languages, I don&#x27;t agree at all. The problem I think a lot about is collaborative text editing. I need to express an insert into a text document - eg, &quot;Insert &#x27;a&#x27; at position X&quot;. The obvious question that comes up is - how should we define &quot;X&quot;? There&#x27;s a mess of options:<p>- UTF-8 byte offset (from the start of the string). (Or UCS2 offset)<p>- Extended grapheme cluster offset<p>- Code point offset<p>You can also use a line&#x2F;column position, but the column position ends up being one of the 3 above.<p>I want collaborative editing protocols to work across multiple programs, running on multiple systems. (Eg, a user is in a web browser on a phone, talking to a server written in rust.)<p>This post says that codepoint offsets are Bad, but in my mind they&#x27;re the only sane answer to this problem.<p>Using byte offsets has two problems:<p>1) You have to pick an encoding, and thats problematic for cross-language compatibility. UTF-8 byte offsets are meaningless in javascript - they&#x27;re slow &amp; expensive to convert. UCS2 offsets are meaningless in rust.<p>2) They make it possible to express invalid data. Inserting in the middle of a codepoint is an invalid operation. I don&#x27;t want to worry about different systems handling that case differently. Using codepoint offsets make it impossible to even represent the idea of inserting in the middle of a character.<p>Using grapheme cluster offsets is problematic because the grapheme clustering rules change all the time. I want editing operations to be readable from any programming language, and across time. Saying &quot;Insert &#x27;a&#x27; at position 20&quot; (measured in grapheme clusters) is ambiguous because &quot;position 20&quot; will drift as unicode&#x27;s grapheme cluster rules change. The result is that old android phones and new iphones can&#x27;t reliably edit a document together. And old editing traces can&#x27;t be reliably replayed.<p>Measuring codepoints is better than the other options because its stable, cross platform and well defined. If you aren&#x27;t aware of those benefits, you aren&#x27;t understanding my use case.