A.3 Does instruction ﬁnetuning improve performance more or less for larger models?
In this paper, we showed that, across a range of model sizes and architectures, instruction ﬁnetuning improves
performance compared with no instruction ﬁnetuning. However, whether the eﬀect of ﬁnetuning increases
with scale is confounded by the fact that the baseline of no instruction ﬁnetuning also gets better with
scale. For example, one could compare the 8B and 540B models as shown in Table 3. Instruction ﬁnetuning
improves the normalized average score across four datasets for 8B model by 15.5% and the 540B model by
9.4%. However, the baseline is also higher for the 540B model, so it could also be relevant to compute the
relative reduction in error rate, which was 18.4% for the 540B model and 16.6% for the 8B model.
Scaling Instruction-Finetuned Language Models