LLM. As one can see in Fig. 11, for 1σ noise sampling, the model is able to solve the task with the new
embeddings with similar performance as with the native embeddings. In case of 2σ, the performance
degrades. Although these are relatively simple single-token prediction tasks, this experiment shows that
LLMs show pattern recognition abilities even when prompted with out-of-distribution continuous input
embeddings. The results are obtained with K = 100, averaged over 3 random seeds when sampling the
token embeddings, 30 instances each, and a context length of 5, 10, or 20 examples. The LLM is the 8B
parameter variant of [55].
LargeLanguageModelsasGeneralPatternMachines