to the model weights, and can be differentiated from in-weights learning, which relies on information
stored in the model weights during LLM training [50] (and can be useful for completion tasks such as
“Abraham Lincoln was born in
”). Chan et al. [50] observes that generalization of in-context learning
can be characterized as more “exemplar-based” (on the basis of similarity to in-context examples [51]),
as opposed to generalization of in-weights learning which tends to be more “rule-based” (on the basis
of minimal features that support category boundaries in the training data [52]). The vast capabilities of
LLMs [39, 53, 54, 55, 56] have been driven by a combination of both forms of learning. In this work, we
are particularly interested in in-context learning, and (depending on the task) using the semantic priors of
numeric tokens to drive capabilities such as sequence completion (Section 5) and improvement (Section 6).
LargeLanguageModelsasGeneralPatternMachines