across models and combines well with other techniques such as UL2R. All these beneﬁts come with relatively
small computational cost compared to model pre-training. For these reasons, we recommend instruction
ﬁnetuning for virtually all pretrained language models.
Scaling Instruction-Finetuned Language Models