title: PaLM 2 Technical Report
summary: We introduce PaLM 2, a new state-of-the-art language model that has better
  multilingual and reasoning capabilities and is more compute-efficient than its
  predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture
  of objectives. Through extensive evaluations on English and multilingual
  language, and reasoning tasks, we demonstrate that PaLM 2 has significantly
  improved quality on downstream tasks across different model sizes, while
  simultaneously exhibiting faster and more efficient inference compared to PaLM.
  This improved efficiency enables broader deployment while also allowing the
  model to respond faster, for a more natural pace of interaction. PaLM 2
  demonstrates robust reasoning capabilities exemplified by large improvements
  over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable
  performance on a suite of responsible AI evaluations, and enables
  inference-time control over toxicity without additional overhead or impact on
  other capabilities. Overall, PaLM 2 achieves state-of-the-art performance
  across a diverse set of tasks and capabilities.
    When discussing the PaLM 2 family, it is important to distinguish between
  pre-trained models (of various sizes), fine-tuned variants of these models, and
  the user-facing products that use these models. In particular, user-facing
  products typically include additional pre- and post-processing steps.
  Additionally, the underlying models may evolve over time. Therefore, one should
  not expect the performance of user-facing products to exactly match the results
  reported in this report.
paper_authors._id: 6465be520e6c7618f6163459
paper_authors.name: Rohan Anil
paper_authors._id: 6465be520e6c7618f616345a
paper_authors.name: Andrew M. Dai
paper_authors._id: 6465be520e6c7618f616345b
paper_authors.name: Orhan Firat
paper_authors._id: 6465be520e6c7618f616345c
paper_authors.name: Melvin Johnson
paper_authors._id: 6465be520e6c7618f616345d
paper_authors.name: Dmitry Lepikhin
paper_authors._id: 6465be520e6c7618f616345e
paper_authors.name: Alexandre Passos
paper_authors._id: 6465be520e6c7618f616345f
paper_authors.name: Siamak Shakeri
paper_authors._id: 6465be520e6c7618f6163460
paper_authors.name: Emanuel Taropa
paper_authors._id: 6465be520e6c7618f6163461
paper_authors.name: Paige Bailey
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-19T13:36:41.368Z
paper_authors._id: 60f300249b7e6dcb17723f3a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1642442497120-60f300249b7e6dcb17723f3a.jpeg
paper_authors.fullname: Paige Bailey
paper_authors.type: user
paper_authors.user: dynamicwebpaige
paper_authors._id: 6465be520e6c7618f6163462
paper_authors.name: Zhifeng Chen
paper_authors._id: 6465be520e6c7618f6163463
paper_authors.name: Eric Chu
paper_authors._id: 6465be520e6c7618f6163464
paper_authors.name: Jonathan H. Clark
paper_authors._id: 6465be520e6c7618f6163465
paper_authors.name: Laurent El Shafey
paper_authors._id: 6465be520e6c7618f6163466
paper_authors.name: Yanping Huang
paper_authors._id: 6465be520e6c7618f6163467
paper_authors.name: Kathy Meier-Hellstern
paper_authors._id: 6465be520e6c7618f6163468
paper_authors.name: Gaurav Mishra
paper_authors._id: 6465be520e6c7618f6163469
paper_authors.name: Erica Moreira
paper_authors._id: 6465be520e6c7618f616346a
paper_authors.name: Mark Omernick
paper_authors._id: 6465be520e6c7618f616346b
paper_authors.name: Kevin Robinson
paper_authors._id: 6465be520e6c7618f616346c
paper_authors.name: Sebastian Ruder
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-19T13:37:32.250Z
paper_authors._id: 624575789dc26c59bae6a19e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1648719221694-noauth.jpeg
paper_authors.fullname: Sebastian Ruder
paper_authors.type: user
paper_authors.user: ruder
paper_authors._id: 6465be520e6c7618f616346d
paper_authors.name: Yi Tay
paper_authors._id: 6465be520e6c7618f616346e
paper_authors.name: Kefan Xiao
paper_authors._id: 6465be520e6c7618f616346f
paper_authors.name: Yuanzhong Xu
paper_authors._id: 6465be520e6c7618f6163470
paper_authors.name: Yujing Zhang
paper_authors._id: 6465be520e6c7618f6163471
paper_authors.name: Gustavo Hernandez Abrego
paper_authors._id: 6465be520e6c7618f6163472
paper_authors.name: Junwhan Ahn
paper_authors._id: 6465be520e6c7618f6163473
paper_authors.name: Jacob Austin
paper_authors._id: 6465be520e6c7618f6163474
paper_authors.name: Paul Barham
paper_authors._id: 6465be520e6c7618f6163475
paper_authors.name: Jan Botha
paper_authors._id: 6465be520e6c7618f6163476
paper_authors.name: James Bradbury
paper_authors._id: 6465be520e6c7618f6163477
paper_authors.name: Siddhartha Brahma
paper_authors._id: 6465be520e6c7618f6163478
paper_authors.name: Kevin Brooks
paper_authors._id: 6465be520e6c7618f6163479
paper_authors.name: Michele Catasta
paper_authors._id: 6465be520e6c7618f616347a
paper_authors.name: Yong Cheng
paper_authors._id: 6465be520e6c7618f616347b
paper_authors.name: Colin Cherry
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-19T13:38:37.532Z
paper_authors._id: 6144bb22207b8aba44e8ec59
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631896530912-6144bb22207b8aba44e8ec59.jpeg
paper_authors.fullname: Colin Cherry
paper_authors.type: user
paper_authors.user: ColinCherry
paper_authors._id: 6465be520e6c7618f616347c
paper_authors.name: Christopher A. Choquette-Choo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-13T07:51:40.132Z
paper_authors._id: 649c5307e89a7650158f4965
paper_authors.avatarUrl: /avatars/3c71b501db27c86ed686ae26ad1391cd.svg
paper_authors.fullname: Christopher A. Choquette-Choo
paper_authors.type: user
paper_authors.user: cchoquette
paper_authors._id: 6465be520e6c7618f616347d
paper_authors.name: Aakanksha Chowdhery
paper_authors._id: 6465be520e6c7618f616347e
paper_authors.name: Clément Crepy
paper_authors._id: 6465be520e6c7618f616347f
paper_authors.name: Shachi Dave
paper_authors._id: 6465be520e6c7618f6163480
paper_authors.name: Mostafa Dehghani
paper_authors._id: 6465be520e6c7618f6163481
paper_authors.name: Sunipa Dev
paper_authors._id: 6465be520e6c7618f6163482
paper_authors.name: Jacob Devlin
paper_authors._id: 6465be520e6c7618f6163483
paper_authors.name: Mark Díaz
paper_authors._id: 6465be520e6c7618f6163484
paper_authors.name: Nan Du
paper_authors._id: 6465be520e6c7618f6163485
paper_authors.name: Ethan Dyer
paper_authors._id: 6465be520e6c7618f6163486
paper_authors.name: Vlad Feinberg
paper_authors._id: 6465be520e6c7618f6163487
paper_authors.name: Fangxiaoyu Feng
paper_authors._id: 6465be520e6c7618f6163488
paper_authors.name: Vlad Fienber
paper_authors._id: 6465be520e6c7618f6163489
paper_authors.name: Markus Freitag
paper_authors._id: 6465be520e6c7618f616348a
paper_authors.name: Xavier Garcia
paper_authors._id: 6465be520e6c7618f616348b
paper_authors.name: Sebastian Gehrmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-19T13:37:19.440Z
paper_authors._id: 618a8d67cabd3c4c8e4481f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1636661214776-618a8d67cabd3c4c8e4481f0.jpeg
paper_authors.fullname: Sebastian Gehrmann
paper_authors.type: user
paper_authors.user: SebGehr
paper_authors._id: 6465be520e6c7618f616348c
paper_authors.name: Lucas Gonzalez
paper_authors._id: 6465be520e6c7618f616348d
paper_authors.name: Guy Gur-Ari
paper_authors._id: 6465be520e6c7618f616348e
paper_authors.name: Steven Hand
paper_authors._id: 6465be520e6c7618f616348f
paper_authors.name: Hadi Hashemi
paper_authors._id: 6465be520e6c7618f6163490
paper_authors.name: Le Hou
paper_authors._id: 6465be520e6c7618f6163491
paper_authors.name: Joshua Howland
paper_authors._id: 6465be520e6c7618f6163492
paper_authors.name: Andrea Hu
paper_authors._id: 6465be520e6c7618f6163493
paper_authors.name: Jeffrey Hui
paper_authors._id: 6465be520e6c7618f6163494
paper_authors.name: Jeremy Hurwitz
paper_authors._id: 6465be520e6c7618f6163495
paper_authors.name: Michael Isard
paper_authors._id: 6465be520e6c7618f6163496
paper_authors.name: Abe Ittycheriah
paper_authors._id: 6465be520e6c7618f6163497
paper_authors.name: Matthew Jagielski
paper_authors._id: 6465be520e6c7618f6163498
paper_authors.name: Wenhao Jia
paper_authors._id: 6465be520e6c7618f6163499
paper_authors.name: Kathleen Kenealy
paper_authors._id: 6465be520e6c7618f616349a
paper_authors.name: Maxim Krikun
paper_authors._id: 6465be520e6c7618f616349b
paper_authors.name: Sneha Kudugunta
paper_authors._id: 6465be520e6c7618f616349c
paper_authors.name: Chang Lan
paper_authors._id: 6465be520e6c7618f616349d
paper_authors.name: Katherine Lee
paper_authors._id: 6465be520e6c7618f616349e
paper_authors.name: Benjamin Lee
paper_authors._id: 6465be520e6c7618f616349f
paper_authors.name: Eric Li
paper_authors._id: 6465be520e6c7618f61634a0
paper_authors.name: Music Li
paper_authors._id: 6465be520e6c7618f61634a1
paper_authors.name: Wei Li
paper_authors._id: 6465be520e6c7618f61634a2
paper_authors.name: YaGuang Li
paper_authors._id: 6465be520e6c7618f61634a3
paper_authors.name: Jian Li
paper_authors._id: 6465be520e6c7618f61634a4
paper_authors.name: Hyeontaek Lim
paper_authors._id: 6465be520e6c7618f61634a5
paper_authors.name: Hanzhao Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-21T09:56:45.561Z
paper_authors._id: 63608df58fb9c2420fff8225
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63608df58fb9c2420fff8225/TZlx0aSH3YxNbyWki65We.jpeg
paper_authors.fullname: Hanzhao Lin
paper_authors.type: user
paper_authors.user: hanzhao
paper_authors._id: 6465be520e6c7618f61634a6
paper_authors.name: Zhongtao Liu
paper_authors._id: 6465be520e6c7618f61634a7
paper_authors.name: Frederick Liu
paper_authors._id: 6465be520e6c7618f61634a8
paper_authors.name: Marcello Maggioni
paper_authors._id: 6465be520e6c7618f61634a9
paper_authors.name: Aroma Mahendru
paper_authors._id: 6465be520e6c7618f61634aa
paper_authors.name: Joshua Maynez
paper_authors._id: 6465be520e6c7618f61634ab
paper_authors.name: Vedant Misra
paper_authors._id: 6465be520e6c7618f61634ac
paper_authors.name: Maysam Moussalem
paper_authors._id: 6465be520e6c7618f61634ad
paper_authors.name: Zachary Nado
paper_authors._id: 6465be520e6c7618f61634ae
paper_authors.name: John Nham
paper_authors._id: 6465be520e6c7618f61634af
paper_authors.name: Eric Ni
paper_authors._id: 6465be520e6c7618f61634b0
paper_authors.name: Andrew Nystrom
paper_authors._id: 6465be520e6c7618f61634b1
paper_authors.name: Alicia Parrish
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-07T20:41:27.369Z
paper_authors._id: 659d71eaf0ae1a57ae140831
paper_authors.avatarUrl: /avatars/b9eb8c70b4fe5dbd0f1555602ae41b65.svg
paper_authors.fullname: Alicia Parrish
paper_authors.type: user
paper_authors.user: avparrish
paper_authors._id: 6465be520e6c7618f61634b2
paper_authors.name: Marie Pellat
paper_authors._id: 6465be520e6c7618f61634b3
paper_authors.name: Martin Polacek
paper_authors._id: 6465be520e6c7618f61634b4
paper_authors.name: Alex Polozov
paper_authors._id: 6465be520e6c7618f61634b5
paper_authors.name: Reiner Pope
paper_authors._id: 6465be520e6c7618f61634b6
paper_authors.name: Siyuan Qiao
paper_authors._id: 6465be520e6c7618f61634b7
paper_authors.name: Emily Reif
paper_authors._id: 6465be520e6c7618f61634b8
paper_authors.name: Bryan Richter
paper_authors._id: 6465be520e6c7618f61634b9
paper_authors.name: Parker Riley
paper_authors._id: 6465be520e6c7618f61634ba
paper_authors.name: Alex Castro Ros
paper_authors._id: 6465be520e6c7618f61634bb
paper_authors.name: Aurko Roy
paper_authors._id: 6465be520e6c7618f61634bc
paper_authors.name: Brennan Saeta
paper_authors._id: 6465be520e6c7618f61634bd
paper_authors.name: Rajkumar Samuel
paper_authors._id: 6465be520e6c7618f61634be
paper_authors.name: Renee Shelby
paper_authors._id: 6465be520e6c7618f61634bf
paper_authors.name: Ambrose Slone
paper_authors._id: 6465be520e6c7618f61634c0
paper_authors.name: Daniel Smilkov
paper_authors._id: 6465be520e6c7618f61634c1
paper_authors.name: David R. So
paper_authors._id: 6465be520e6c7618f61634c2
paper_authors.name: Daniel Sohn
paper_authors._id: 6465be520e6c7618f61634c3
paper_authors.name: Simon Tokumine
paper_authors._id: 6465be520e6c7618f61634c4
paper_authors.name: Dasha Valter
paper_authors._id: 6465be520e6c7618f61634c5
paper_authors.name: Vijay Vasudevan
paper_authors._id: 6465be520e6c7618f61634c6
paper_authors.name: Kiran Vodrahalli
paper_authors._id: 6465be520e6c7618f61634c7
paper_authors.name: Xuezhi Wang
paper_authors._id: 6465be520e6c7618f61634c8
paper_authors.name: Pidong Wang
paper_authors._id: 6465be520e6c7618f61634c9
paper_authors.name: Zirui Wang
paper_authors._id: 6465be520e6c7618f61634ca
paper_authors.name: Tao Wang
paper_authors._id: 6465be520e6c7618f61634cb
paper_authors.name: John Wieting
paper_authors._id: 6465be520e6c7618f61634cc
paper_authors.name: Yuhuai Wu
paper_authors._id: 6465be520e6c7618f61634cd
paper_authors.name: Kelvin Xu
paper_authors._id: 6465be520e6c7618f61634ce
paper_authors.name: Yunhan Xu
paper_authors._id: 6465be520e6c7618f61634cf
paper_authors.name: Linting Xue
paper_authors._id: 6465be520e6c7618f61634d0
paper_authors.name: Pengcheng Yin
paper_authors._id: 6465be520e6c7618f61634d1
paper_authors.name: Jiahui Yu
paper_authors._id: 6465be520e6c7618f61634d2
paper_authors.name: Qiao Zhang
paper_authors._id: 6465be520e6c7618f61634d3
paper_authors.name: Steven Zheng
paper_authors._id: 6465be520e6c7618f61634d4
paper_authors.name: Ce Zheng
paper_authors._id: 6465be520e6c7618f61634d5
paper_authors.name: Weikang Zhou
paper_authors._id: 6465be520e6c7618f61634d6
paper_authors.name: Denny Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-20T18:28:40.639Z
paper_authors._id: 65d4d62d493aeaf0bb295f1b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vXMxaoWwFziul8TqIhQw8.jpeg
paper_authors.fullname: Denny Zhou
paper_authors.type: user
paper_authors.user: dennyzhou
paper_authors._id: 6465be520e6c7618f61634d7
paper_authors.name: Slav Petrov
paper_authors._id: 6465be520e6c7618f61634d8
paper_authors.name: Yonghui Wu
paper_title: PaLM 2 Technical Report
paper_summary: We introduce PaLM 2, a new state-of-the-art language model that has better
  multilingual and reasoning capabilities and is more compute-efficient than its
  predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture
  of objectives. Through extensive evaluations on English and multilingual
  language, and reasoning tasks, we demonstrate that PaLM 2 has significantly
  improved quality on downstream tasks across different model sizes, while
  simultaneously exhibiting faster and more efficient inference compared to PaLM.
  This improved efficiency enables broader deployment while also allowing the
  model to respond faster, for a more natural pace of interaction. PaLM 2
  demonstrates robust reasoning capabilities exemplified by large improvements
  over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable
  performance on a suite of responsible AI evaluations, and enables
  inference-time control over toxicity without additional overhead or impact on
  other capabilities. Overall, PaLM 2 achieves state-of-the-art performance
  across a diverse set of tasks and capabilities.
    When discussing the PaLM 2 family, it is important to distinguish between
  pre-trained models (of various sizes), fine-tuned variants of these models, and
  the user-facing products that use these models. In particular, user-facing
  products typically include additional pre- and post-processing steps.
  Additionally, the underlying models may evolve over time. Therefore, one should
  not expect the performance of user-facing products to exactly match the results
  reported in this report.
paper_ai_keywords: Transformer-based mixture of objectives multilingual language reasoning tasks downstream tasks inference BIG-Bench responsible AI evaluations toxicity user-facing products pre- and post-processing steps