PaLM 2 outperforms PaLM across all datasets and achieves results competitive with GPT-4. On the multilingual
XCOPA dataset, PaLM 2 achieves particularly strong improvements on under-represented languages such as Swahili,
Quechua, and Haitian and establishes a new state of the art even without chain-of-thought prompting (Wei et al., 2022)
(see Appendix A.3 for the detailed results). On BIG-Bench Hard, PaLM 2 outperforms PaLM on every task, often by a
large margin. We discuss improvements on the challenging BIG-Bench Hard tasks below.

5SOTA approaches for DROP and CSQA employ dataset-speciﬁc strategies.
6WinoGrande, ARC, DROP, and COPA are contained in the Flan mixture (Chung et al., 2022) used to instruction-tune PaLM 2. Training data
(but not dev or test data) has thus been seen by the model as part of a very large mixture. We are not aware to what extent other models have been
exposed to the training data.

13
PaLM 2 Technical Report