Results are shown in Table 8. PaLM 2-S* outperforms PaLM-540B-Coder on all benchmarks, often by a signiﬁcant
margin (e.g. ARCADE), despite being dramatically smaller, cheaper, and faster to serve.

Multilingual Evaluation We also evaluate PaLM 2-S*’s multilingual coding abilities using BabelCode (Orlanski
et al., 2023) which translates HumanEval into a variety of other programming languages, including high-resource
languages like C++, Java, and Go and low-resource languages like Haskell and Julia. The PaLM 2 code training data is
signiﬁcantly more multilingual than PaLM’s, which we hope yields signiﬁcant gains on coding evals. Figure 6 shows
PaLM 2-S*’s results compared to the original PaLM models. We show an example of multilingual program generation
in Figure 7.
PaLM 2 Technical Report