Table 6: Performance of the non-instruction-tuned
175B parameter GPT-3 model (davinci) in the zero-shot
setting, which we propose as the setting to evaluate
tasks in the absence of in-context learning. For a task to
be considered emergent (Emg.), models must perform
above the baseline (> Base.) and the performance of
the larger models must not be predictable based on that
of smaller models (Pred.). Results marked with a star
indicate that they are not significant.

Recall that the definition of emergence, as stated
by Wei et al. (2022b), requires LLMs to perform a
task above the baseline and do so in a manner that
cannot be predicted based on the performance of
smaller models. Despite the criticisms of the use of
a discrete evaluation metric (Section 2), we delib-
erately use BERTScore accuracy, a discrete metric.
This choice is driven by our desire to provide every
opportunity to identify emergent abilities.
AreEmergentAbilitiesinLarge Language Models just In-Context