3 Methods

To answer the research questions presented in Sec-
tion 1.4, we examine each of the following: 1)
The impact of removing the factor of in-context
learning by working in the zero-shot setting and re-
moving the factor of instruction tuning by working
with non-instruction-tuned models. 2) The inter-
play between in-context learning and instruction
tuning to determine whether ‘reasoning’ or simi-
lar explanations are necessary to explain the ad-
ditional capabilities of instruction-tuned models.
We achieved this by comparing the capabilities of
models in the absence of instruction-tuning and

Figure 2: The process of instruction tuning. Source
Wei et al. (2022a).
AreEmergentAbilitiesinLarge Language Models just In-Context