Abstract: We observe that pre-trained large language models (LLMs) are capable of au-
toregressively completing complex token sequences—from arbitrary ones procedurally
generated by probabilistic context-free grammars (PCFG), to more rich spatial pat-
terns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark,
prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be
partially retained even when the sequences are expressed using tokens randomly sampled
from the vocabulary. These results suggest that without any additional training, LLMs
can serve as general sequence modelers, driven by in-context learning. In this work, we
investigate how these zero-shot capabilities may be applied to problems in robotics—
from extrapolating sequences of numbers that represent states over time to complete
simple motions, to least-to-most prompting of reward-conditioned trajectories that can
LargeLanguageModelsasGeneralPatternMachines