4 Finetuning with chain-of-thought annotations

The goal of Flan ﬁnetuning is to produce an improved checkpoint across a range of evaluations, which
includes multi-step reasoning ability in addition to traditional NLP tasks. In this section we explore the
eﬀect of including chain-of-thought (CoT) data in the instruction ﬁnetuning mixture. First, we show that the
improved reasoning abilities of Flan-PaLM surprass prior models on several benchmarks. Then, we ablate
CoT ﬁnetuning data and show that whereas instruction ﬁnetuning without CoT actually degrades reasoning
ability, including just nine CoT datasets improves performance on all evaluations. Finally, we show that CoT
ﬁnetuning unlocks zero-shot reasoning via “let’s think step-by-step” on challenging BIG-Bench tasks.
Scaling Instruction-Finetuned Language Models