Show HN: ChatGPT Plugins are a security nightmare
I wonder if a lot of those &quot;injection&quot; problems could be overcome by introducing a distinction between the different types of input and output already at the token level.<p>E.g. imagine that every token that an LLM inputs or outputs would be associated with a &quot;color&quot; or &quot;channel&quot;, which corresponds to the token&#x27;s source or destination:<p>- &quot;red&quot;: tokens input by the user, i.e. the initial prompt and subsequent replies.<p>- &quot;green&quot;: answers from the LLM to the user, i.e. everything the user sees as textual output on the screen.<p>- &quot;blue&quot;: instructions from the LLM to a plugin: database queries, calculations, web requests, etc.<p>- &quot;yellow&quot;: replies from the plugin back to the LLM.<p>- &quot;purple&quot;: the initial system prompt.<p>The point is that each (word, color) combination constitutes a separate token; i.e. if your &quot;root&quot; token dictionary was as follows:<p>hello -&gt; 0001;
world -&gt; 0002;<p>then the &quot;colorized&quot; token dictionary would be the cross product of the root and each color combination:<p>hello (red) -&gt; 0001;
hello (green) -&gt; 0002;
...
world (red) -&gt; 0006;
world (green) -&gt; 0007;
...<p>likewise, because the model considers &quot;hello (red)&quot; and &quot;hello (blue)&quot; two different tokens, it also has two different sets of weights for those tokens and hopefully much less risk of confusing one kind of token with the other.<p>With some luck, you don&#x27;t have to use 5 x the amount of compute and training data for training: You might be able to take an &quot;ordinary&quot; model, trained on non-colored tokens, then copy the weights four times and finetune the resulting &quot;expanded&quot; model on a colored corpus.<p>Likewise, because the model should only ever predict &quot;green&quot; or &quot;blue&quot; tokens, any output neuron that correspond only to &quot;red&quot;, &quot;yellow&quot; or &quot;purple&quot; tokens can be removed from the model.