30,000
20,000
100,000
5,000

46.0%
27.9%
1.8%
24.2%

52%
15%
3%
30%

Table 23: Maximum example cap and proportion rates applied to the underlying mixtures. Proportion A
was used in scaling and the ablation sections in Section 3 and Section 4. The rest of the experiments used
Proportion B, since Proportion A signaled that the T0 mixture was good for performance.

For balancing the four mixtures, we use the relative weights ensuring that none of the underlying tasks is
repeated more than once. For the scaling and ablation experiments in Section 3 and Section 4, respectively,
we used the mixture proportions in Table 23, Proportion A. Based on these experiments (speciﬁcally, strong
gains from T0-SF), we updated the mixture proportions to the Proportion B values in Table 23 and ﬁnetuned
the rest of the models.

46

F Finetuning Data Card

Dataset Owners. ﬂan-core@google.com

We present the data card, following the format proposed by Pushkarna et al. (2021).
Scaling Instruction-Finetuned Language Models