ChatGPT, Rot13, and Daniel Kahneman
It&#x27;s really not so complicated. This is just an issue with text tokenization, and the fact that the learning model never actually sees the raw input bytes.<p>All modern LLMs use a tokenizer to convert a sequence of bytes into a sequence of tokens. Short, common words like &quot;the&quot; and &quot;why&quot; are represented as single tokens, while longer and less-common words are represented by multiple tokens. For example, the word &quot;fantastic&quot; is three tokens (&quot;f&quot;, &quot;ant&quot;, &quot;astic&quot;).<p>Each of these tokens is assigned an arbitrary integer value (&quot;fantastic&quot; becomes [69, 415, 3477]) and then those integer values are used to lookup embedding vectors for each word.<p>Each embedding vector represents the <i>MEANING</i> of the tokens, by plotting them into a 4096-dimensional vector-space. At runtime, the model looks up each token ID in a dictionary and finds its embedding vector.<p>For the word &quot;fantastic&quot;, those embedding vectors might look something like this:<p><pre><code>  &quot;f&quot;        (69) = [  0.123,  0.456, ...etc...  0.789, -0.890 ]
  &quot;ant&quot;     (415) = [  0.111, -0.222, ...etc...  0.333, -0.444 ]
  &quot;astic&quot;  (3477) = [ -0.101,  0.202, ...etc... -0.303,  0.404 ]
</code></pre>
All of these vectors are assembled into a matrix, and then passed into the layers of neural network, where the actual training&#x2F;inference occurs.<p>So the language-model has NO IDEA how any of the words are spelled, because the tokenization (and embedding vector lookup) happens as a pre-processing step, outside the bounds of the learning algorithm.<p>If you want a LLM to understand spelling, you have to include exhaustive spelling information in its training data. For example:<p><pre><code>  &quot;The word &#x27;fantastic&#x27; is spelled f-a-n-t-a-s-t-i-c.&quot;
  &quot;The word &#x27;FANTASTIC&#x27; is spelled F-A-N-T-A-S-T-I-C.&quot;
  ...etc...
</code></pre>
And even then, even with 100k+ English words all spelled out in your training data, you&#x27;d be hard-pressed to infer any ROT-13 tokens in your output data, because the learning model has probably never seen a token like &quot;qvq&quot; or &quot;pebff&quot;.<p>You can play with the GPT tokenizer directly here:<p><a href="https:&#x2F;&#x2F;beta.openai.com&#x2F;tokenizer" rel="nofollow">https:&#x2F;&#x2F;beta.openai.com&#x2F;tokenizer</a><p>It will show you the tokenization of any block of text, and the token IDs of the resultant tokens. It&#x27;s very handy if you spend much time working with GPT-3 (or any other modern language-model!)