7

models at different scales instruction-tuned to dif-
ferent extents. 3) If each of the tasks typically used
to evaluate models assesses functional linguistic
abilities, formal linguistic abilities, and whether
the task can be memorised, which we evaluated
through a manually examining each task.

In order to assess the true extent of emergent
abilities in LLMs, we meticulously structure our
experiments to minimise any factors that might cre-
ate the appearance of emergence. Our experimental
design specifically focuses on assessing models in
a manner that does not trigger in-context learning.
It is important to note that instruction tuning, which
entails training models to translate instructions into
exemplars (refer to Section 2), could potentially
trigger in-context learning. Therefore, we take pre-
cautions to account for the possibility that instruc-
tion tuning triggers in-context learning by using
non-instruction-tuned models.
AreEmergentAbilitiesinLarge Language Models just In-Context