ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2305.10403
22
1136

PaLM 2 Technical Report

17 May 2023
Rohan Anil
Andrew M. Dai
Orhan Firat
Melvin Johnson
Dmitry Lepikhin
Alexandre Passos
Siamak Shakeri
Emanuel Taropa
Paige Bailey
Z. Chen
Eric Chu
J. Clark
Laurent El Shafey
Yanping Huang
Kathy Meier-Hellstern
Gaurav Mishra
Erica Moreira
Mark Omernick
Kevin Robinson
Sebastian Ruder
Yi Tay
Kefan Xiao
Yuanzhong Xu
Yujing Zhang
Gustavo Hernández Ábrego
Junwhan Ahn
Jacob Austin
P. Barham
Jan A. Botha
James Bradbury
Siddhartha Brahma
K. Brooks
Michele Catasta
Yongzhou Cheng
Colin Cherry
Christopher A. Choquette-Choo
Aakanksha Chowdhery
Clément Crepy
Shachi Dave
Mostafa Dehghani
Sunipa Dev
Jacob Devlin
Mark Díaz
Nan Du
Ethan Dyer
Vladimir Feinberg
Fan Feng
Vlad Fienber
Markus Freitag
Xavier Garcia
Sebastian Gehrmann
Lucas González
Guy Gur-Ari
Steven Hand
Hadi Hashemi
Le Hou
Joshua Howland
A. Hu
Jeffrey Hui
Jeremy Hurwitz
Michael Isard
Abe Ittycheriah
Matthew Jagielski
W. Jia
Kathleen Kenealy
M. Krikun
Sneha Kudugunta
Chang Lan
Katherine Lee
Benjamin Lee
Eric Li
Mu-Li Li
Wei Li
Yaguang Li
Jun Yu Li
Hyeontaek Lim
Han Lin
Zhong-Zhong Liu
Frederick Liu
Marcello Maggioni
Aroma Mahendru
Joshua Maynez
Vedant Misra
Maysam Moussalem
Zachary Nado
John Nham
Eric Ni
A. Nystrom
Alicia Parrish
Marie Pellat
M. Polacek
Oleksandr Polozov
Reiner Pope
Siyuan Qiao
Emily Reif
Bryan Richter
Parker Riley
Alex Castro-Ros
Aurko Roy
Brennan Saeta
Rajkumar Samuel
Renee Shelby
Ambrose Slone
D. Smilkov
David R. So
Daniela Sohn
Simon Tokumine
Dasha Valter
Vijay Vasudevan
Kiran Vodrahalli
Xuezhi Wang
Pidong Wang
Zirui Wang
Tao Wang
John Wieting
Yuhuai Wu
Ke Xu
Yunhan Xu
L. Xue
Pengcheng Yin
Jiahui Yu
Qiaoling Zhang
Steven Zheng
Ce Zheng
Wei Zhou
Denny Zhou
Slav Petrov
Yonghui Wu
    ReLM
    LRM
ArXivPDFHTML
Abstract

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

View on arXiv
Comments on this paper