CoT ﬁnetuning is critical for reasoning abilities. Although previous instruction ﬁnetuning work has shown
that ﬁnetuning on non-CoT tasks improves performance on unseen non-CoT tasks, we ﬁnd that this actually
leads to degraded performance on CoT tasks. As a solution for this degraded CoT performance, we jointly
ﬁnetune on both non-CoT and CoT data (Section 4). This joint ﬁnetuning enables substantially better CoT
performance while maintaining performance on non-CoT tasks, allowing a single model to do well on all
evaluations. Whereas prior work has shown that CoT ﬁnetuning improves performance on tasks that were
held-in during ﬁnetuning (Ling et al., 2017; Cobbe et al., 2021; Zelikman et al., 2022; Huang et al., 2022,
inter alia), we showed that CoT ﬁnetuning a large model improves performance on held-out tasks while
maintaining performance improvements for non-CoT tasks.
Instruction ﬁnetuning generalizes across models. In Section 5, we observed the generality of instruction
Scaling Instruction-Finetuned Language Models