An important point is that Figure 5-left also shows that it is critical to ﬁnetune on some CoT examples in
order to keep such reasoning abilities, because ﬁnetuning on only non-CoT degrades performance on CoT by
a substantial amount, as shown by the green line. This degradation may be surprising in light of multiple
prior studies ﬁnding that instruction ﬁnetuning improves performance on unseen tasks (Wei et al., 2021;
Sanh et al., 2021; Wang et al., 2019a; Min et al., 2022, inter alia). However, the previous work only evaluated
held-out NLP tasks (e.g., ﬁnetuning on all tasks except sentiment analysis, and then evaluating on sentiment
analysis benchmarks), and the prior models were generally too small for successful CoT reasoning. Together,
one might interpret this ablation as the following: instruction ﬁnetuning improves unseen tasks when the
unseen tasks are in the same prompting paradigm as the ﬁnetuning tasks (i.e., non-CoT or CoT). Hence, both
Scaling Instruction-Finetuned Language Models