Potential misgendering harms We measure PaLM 2 on failures that can lead to potential misgendering harms in
zero-shot translation. When translating into English, we ﬁnd stable performance on PaLM 2 compared to PaLM, with
small improvements on worst-case disaggregated performance across 26 languages. When translating out of English into
13 languages, we evaluate gender agreement and translation quality with human raters. Surprsingly, we ﬁnd that even in
the zero-shot setting PaLM 2 outperforms PaLM and Google Translate on gender agreement in three high-resource
languages: Spanish, Polish and Portuguese. We observe lower gender agreement scores when translating into Telugu,
Hindi and Arabic with PaLM 2 as compared to PaLM. See Appendix E.5 for results and analysis.

4.6 Natural language generation
PaLM 2 Technical Report