9

Prev. Emergent Competence type Memorisable (of 50)

Task
Causal judgement
English Proverbs
Implicatures
Nonsense words grammar
Rhyming
Tracking shuffled objects
Commonsense QA (Talmor et al., 2019)
GSM8K
Analytic entailment
Codenames
Common morpheme
Fact checker
Figure of speech detection
Hindu knowledge
Logical deduction
Misconceptions
Modified arithmetic
Phrase relatedness
Physical intuition
Social IQa (Sap et al., 2019)
Strange stories
Strategy QA (Geva et al., 2021)

No
No
No
No
No
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes

Functional
Functional
Functional

Formal
Formal

Functional
Functional
Functional
Functional
Functional

Formal

Functional
Functional
Functional
Functional
Functional
Functional
Functional
Functional
Functional
Functional
Functional

0/50
0/50
0/50
38/50
50/50
0/50
3/50
0/50
4/50
0/50
0/50
50/50
0/50
50/50
0/50
50/50
0/50
50/50
50/50
0/50
0/50
27/50
AreEmergentAbilitiesinLarge Language Models just In-Context