In addition, we evaluate six selected models
from the LLaMA and Falcon families (See Sec-
tion 3.1), on four of the 22 tasks chosen earlier.
We pick these four tasks ensuring that two have
been previously identified as emergent (Logical De-
ductions and Social IQA) and the other two have
been determined to be non-emergent (GSM8K and
Tracking Shuffled Objects). Once again we test
these using the closed and adversarial prompting
strategies and run each experiment thrice to account
for variance. Given the variable number of options
associated with some of the tasks under evaluation,
we construct the baseline for each task by randomly
selecting options for questions in that task multiple
times and finding an average score.

4 Results: Emergent Abilities in the

absence of In-context Learning
AreEmergentAbilitiesinLarge Language Models just In-Context