Gemma: Open Models Based on Gemini Research and Technology
<|reference_start|>Gemma: Open Models Based on Gemini Research and Technology: This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations.<|reference_end|>
@article{gemma team2024gemma:,
    title={Gemma: Open Models Based on Gemini Research and Technology},
    author={Gemma Team: Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya
  Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi`ere, Mihir Sanjay
  Kale, Juliette Love, Pouya Tafti, L'eonard Hussenot, Pier Giuseppe Sessa,
  Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros,
  Ambrose Slone, Am'elie H'eliou, Andrea Tacchetti, Anna Bulanova, Antonia
  Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A.
  Choquette-Choo, Cl'ement Crepy, Daniel Cer, Daphne Ippolito, David Reid,
  Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker,
  George-Christian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian
  Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski,
  Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan
  Ferret, Justin Chiu, Justin Mao-Jones, Katherine Lee, Kathy Yu, Katie
  Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciej
  Miku{l}a, Mateo Wirth, Michael Sharman, Nikolai Chinaev, Nithum Thain,
  Olivier Bachem, Oscar Chang, Oscar Wahltinez, Paige Bailey, Paul Michel,
  Petko Yotov, Rahma Chaabouni, Ramona Comanescu, Reena Jana, Rohan Anil, Ross
  McIlroy, Ruibo Liu, Ryan Mullins, Samuel L Smith, Sebastian Borgeaud, Sertan
  Girgin, Sholto Douglas, Shree Pandya, Siamak Shakeri, Soham De, Ted Klimenko,
  Tom Hennigan, Vlad Feinberg, Wojciech Stokowiec, Yu-hui Chen, Zafarali Ahmed,
  Zhitao Gong, Tris Warkentin, Ludovic Peran, Minh Giang, Cl'ement Farabet,
  Oriol Vinyals, Jeff Dean, Koray Kavukcuoglu, Demis Hassabis, Zoubin
  Ghahramani, Douglas Eck, Joelle Barral, Fernando Pereira, Eli Collins, Armand
  Joulin, Noah Fiedel, Evan Senter, Alek Andreev, Kathleen Kenealy},
    journal={arXiv preprint arXiv:2403.08295},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.08295},
    primaryClass={cs.CL cs.AI}
}