4 Results: Emergent Abilities in the ab-

sence of In-context Learning
4.1 Overview of Results . . . . . . . .
4.2 Model Performance . . . . . . . .
4.3 Emergent Abilities
. . . . . . . .
4.4 Other Model Families . . . . . . .
4.5 Possible Biasing Factors
. . . . .
4.5.1 Manual Evaluation . . . .
4.5.2
Interim Summary and Key Insights

12
13
13
13
15
15
16
Shortened output Generation 16
17

4.6

5 Results: Instruction Tuning as Efficient

18
In-Context Learning
18
5.1 Results Overview . . . . . . . . .
19
5.2
In-Context in T5 . . . . . . . . .
5.3 Comparative Analysis of Initial Tasks 19
5.4 Comparative Analysis of Instruc-
tion Following . . . . . . . .
. . .
Interim Summary and Key Insights
5.5.1 Task Solvability . . . . . .

21
23
23

5.5

6

Implications and Broader Context

7 Conclusions and Future Work

4
4
5

5

7
8
8
9
9
11
11
12

23

24

24

8 Limitations

1

1

Introduction, Motivation and Context
AreEmergentAbilitiesinLarge Language Models just In-Context