2.2 Downstream metric evaluations

To study the effect of choosing an optimal number of parameters and training tokens given a ﬁxed compute cost of
1× 1022 FLOPs, we train several models on the same pre-training mixture, sampled according to each model size to be
trained on the given FLOPs. It is important to note that the selected model sizes, architecture and the training mixture
were only used for the study of the scaling laws.
PaLM 2 Technical Report