Gemma 3 Technical Report
We introduce Gemma 3, a multimodal addition to the Gemma family of
lightweight open models, ranging in scale from 1 to 27 billion parameters. This
version introduces vision understanding abilities, a wider coverage of
languages and longer context - at least 128K tokens. We also change the
architecture of the model to reduce the KV-cache memory that tends to explode
with long context. This is achieved by increasing the ratio of local to global
attention layers, and keeping the span on local attention short. The Gemma 3
models are trained with distillation and achieve superior performance to Gemma
2 for both pre-trained and instruction finetuned versions. In particular, our
novel post-training recipe significantly improves the math, chat,
instruction-following and multilingual abilities, making Gemma3-4B-IT
competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro
across benchmarks. We release all our models to the community.
67e5267538e4d1444c8bb953
Gemma Team
67e5267538e4d1444c8bb954
Aishwarya Kamath
claimed_verified
2025-03-28T16:27:50.620Z
67c9cc34bebf5d13bf6c728a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/j0BRlusRUUSO4fQOLRTlv.png
Aishwarya Kamath
user
ashkamathwork
67e5267538e4d1444c8bb955
Johan Ferret
admin_assigned
2025-03-28T16:28:03.533Z
65afb7dbdd6bdfd73cd8e609
/avatars/b21069bc2d7ee4cc1508008e3c8ade64.svg
Johan Ferret
user
ferretj
67e5267538e4d1444c8bb956
Shreya Pathak
admin_assigned
2025-03-28T16:28:09.908Z
667125edfc615a6257be29bc
/avatars/c1b191d6443b7a4ac52866a746ca5717.svg
Shreya Pathak
user
shreyapathak
67e5267538e4d1444c8bb957
Nino Vieillard
67e5267538e4d1444c8bb958
Ramona Merhej
admin_assigned
2025-03-28T16:28:19.392Z
67c9bfe6ebe5592f1f05d966
/avatars/0627e638ff2b09c28fae60d41e290235.svg
Ramona Merhej
user
rmerhej
67e5267538e4d1444c8bb959
Sarah Perrin
admin_assigned
2025-03-28T16:28:27.391Z
64da52ea322a5774e083e10a
/avatars/ea8f1ec6ecec28db24438b499daa6cca.svg
Sarah Perrine
user
7quidifYpaema
67e5267538e4d1444c8bb95a
Tatiana Matejovicova
claimed_verified
2025-03-28T14:22:30.144Z
66608363906b039ad6515b14
https://cdn-avatars.huggingface.co/v1/production/uploads/66608363906b039ad6515b14/W-pBJo9eCrLq0tRiu3CC4.jpeg
Tatiana Matejovicova
user
tatianama1
67e5267538e4d1444c8bb95b
Alexandre Ramé
admin_assigned
2025-03-28T16:28:35.516Z
63c94ede00104ea998de19a6
/avatars/273959d87f0c67747588cf0700d64039.svg
Alexandre Rame
user
alexrame
67e5267538e4d1444c8bb95c
Morgane Rivière
admin_assigned
2025-03-28T16:28:41.702Z
66953d345d5615345518ca31
https://cdn-avatars.huggingface.co/v1/production/uploads/66953d345d5615345518ca31/rIpTxlrMxVNB9Td9fJijl.png
Morgane Riviere
user
morugan
67e5267538e4d1444c8bb95d
Louis Rouillard
67e5267538e4d1444c8bb95e
Thomas Mesnard
admin_assigned
2025-03-28T16:28:51.842Z
6639adab8eb8c9c62ac3fb15
/avatars/83133b04922143be1d8a1fb71fb2938f.svg
Thomas Mesnard
user
thomasmesnard
67e5267538e4d1444c8bb95f
Geoffrey Cideron
67e5267538e4d1444c8bb960
Jean-bastien Grill
67e5267538e4d1444c8bb961
Sabela Ramos
67e5267538e4d1444c8bb962
Edouard Yvinec
admin_assigned
2025-03-28T16:29:08.048Z
65a044a9da292edc45176090
/avatars/8440f76c5cf041565a766794c1f432e8.svg
Edouard YVINEC
user
eyvinec
67e5267538e4d1444c8bb963
Michelle Casbon
admin_assigned
2025-03-28T16:29:16.305Z
662806c1167ee93ec8e503a4
/avatars/8dcf47f91328a38833636068686a8a5a.svg
Michelle Casbon
user
michellecasbon
67e5267538e4d1444c8bb964
Etienne Pot
admin_assigned
2025-03-28T16:29:22.103Z
5eed6a0e73936372ea202735
/avatars/a2bfd7c4e403ac42737c2dd285d63a21.svg
Etienne POT
user
conchylicultor
67e5267538e4d1444c8bb965
Ivo Penchev
67e5267538e4d1444c8bb966
Gaël Liu
67e5267538e4d1444c8bb967
Francesco Visin
67e5267538e4d1444c8bb968
Kathleen Kenealy
admin_assigned
2025-03-28T16:29:48.032Z
666b337c3fc0651814997536
/avatars/4d6198300c286c5144794bfd2d1fa653.svg
Kathleen Kenealy
user
kathleenatgoogle
67e5267538e4d1444c8bb969
Lucas Beyer
admin_assigned
2025-03-28T16:29:57.656Z
642d334ff65714b4585f2de4
https://cdn-avatars.huggingface.co/v1/production/uploads/642d334ff65714b4585f2de4/gxBynq5KyoUP0VlAQD3-w.jpeg
Lucas Beyer
user
giffmana
67e5267538e4d1444c8bb96a
Xiaohai Zhai
67e5267538e4d1444c8bb96b
Anton Tsitsulin
admin_assigned
2025-03-28T16:30:16.674Z
63ab8e5c8a357d5057ae659e
/avatars/0eed94fcdb6d7c86259bc5b72b5bd41e.svg
Anton Tsitsulin
user
xgfs
67e5267538e4d1444c8bb96c
Robert Busa-Fekete
admin_assigned
2025-03-28T16:30:24.407Z
6744668747d61b15da8fd345
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/j3zeutPTmRfi5167_Bi47.png
Robert Busa-Fekete
user
busarobi
67e5267538e4d1444c8bb96d
Alex Feng
67e5267538e4d1444c8bb96e
Noveen Sachdeva
67e5267538e4d1444c8bb96f
Benjamin Coleman
admin_assigned
2025-03-28T16:30:44.167Z
665a131a08cba81fb74d745a
/avatars/fc7e216b5abb556762edf4fcdbd9aad0.svg
Benjamin Coleman
user
BenjaminColeman305
67e5267538e4d1444c8bb970
Yi Gao
67e5267538e4d1444c8bb971
Basil Mustafa
admin_assigned
2025-03-28T16:30:51.090Z
63cfcad6e23b90128c66685c
https://cdn-avatars.huggingface.co/v1/production/uploads/1674562206311-noauth.png
Basil Mustafa
user
BasilMustafa
67e5267538e4d1444c8bb972
Iain Barr
67e5267538e4d1444c8bb973
Emilio Parisotto
67e5267538e4d1444c8bb974
David Tian
67e5267538e4d1444c8bb975
Matan Eyal
67e5267538e4d1444c8bb976
Colin Cherry
admin_assigned
2025-03-28T16:31:12.166Z
6144bb22207b8aba44e8ec59
https://cdn-avatars.huggingface.co/v1/production/uploads/1631896530912-6144bb22207b8aba44e8ec59.jpeg
Colin Cherry
user
ColinCherry
67e5267538e4d1444c8bb977
Jan-Thorsten Peter
admin_assigned
2025-03-28T16:31:20.061Z
64cd2ae2e8df1f66dd2936cf
/avatars/3910fa4414961e84a8bd3f844929732b.svg
Jan-Thorsten Peter
user
jtpeter
67e5267538e4d1444c8bb978
Danila Sinopalnikov
admin_assigned
2025-03-28T16:31:27.509Z
667a6ddc400b4b8da62488f1
/avatars/65772bd92bbbe832eeb51298a231f57a.svg
Danila Sinopalnikov
user
sinopalnikov
67e5267538e4d1444c8bb979
Surya Bhupatiraju
admin_assigned
2025-03-28T16:31:35.728Z
65d7644d19bf1e60614dee6e
/avatars/422a8d28fa9d9ff2c3ec1406352cb346.svg
Surya Bhupatiraju
user
suryabhupa
67e5267538e4d1444c8bb97a
Rishabh Agarwal
67e5267538e4d1444c8bb97b
Mehran Kazemi
67e5267538e4d1444c8bb97c
Dan Malkin
67e5267538e4d1444c8bb97d
Ravin Kumar
67e5267538e4d1444c8bb97e
David Vilar
67e5267538e4d1444c8bb97f
Idan Brusilovsky
67e5267538e4d1444c8bb980
Jiaming Luo
67e5267538e4d1444c8bb981
Andreas Steiner
67e5267538e4d1444c8bb982
Abe Friesen
67e5267538e4d1444c8bb983
Abhanshu Sharma
67e5267538e4d1444c8bb984
Abheesht Sharma
67e5267538e4d1444c8bb985
Adi Mayrav Gilady
admin_assigned
2025-03-28T16:33:14.006Z
65ed6b3aada92a05d37a22ab
/avatars/c4139988303da59f92729d0d2e225c9f.svg
Adi Mayrav Gilady
user
AdiM1234
67e5267538e4d1444c8bb986
Adrian Goedeckemeyer
67e5267538e4d1444c8bb987
Alaa Saade
67e5267538e4d1444c8bb988
Alex Feng
67e5267538e4d1444c8bb989
Alexander Kolesnikov
67e5267538e4d1444c8bb98a
Alexei Bendebury
67e5267538e4d1444c8bb98b
Alvin Abdagic
admin_assigned
2025-03-28T16:32:35.080Z
66aaa23c08609915b3855f0b
/avatars/6e6c105d921c41477b61af7a4a1b6495.svg
Alvin Abdagic
user
aabdagic
67e5267538e4d1444c8bb98c
Amit Vadi
admin_assigned
2025-03-28T16:32:27.975Z
67b7632d87214a967bbe1731
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/595EBwH8GzwGBH-nN2k4o.png
Amit Vadi
user
avadi
67e5267538e4d1444c8bb98d
András György
67e5267538e4d1444c8bb98e
André Susano Pinto
67e5267538e4d1444c8bb98f
Anil Das
67e5267538e4d1444c8bb990
Ankur Bapna
67e5267538e4d1444c8bb991
Antoine Miech
67e5267538e4d1444c8bb992
Antoine Yang
67e5267538e4d1444c8bb993
Antonia Paterson
67e5267538e4d1444c8bb994
Ashish Shenoy
67e5267538e4d1444c8bb995
Ayan Chakrabarti
67e5267538e4d1444c8bb996
Bilal Piot
67e5267538e4d1444c8bb997
Bo Wu
67e5267538e4d1444c8bb998
Bobak Shahriari
67e5267538e4d1444c8bb999
Bryce Petrini
67e5267538e4d1444c8bb99a
Charlie Chen
admin_assigned
2025-03-28T16:33:23.694Z
61d3e5e1733d3a83ecd1bbc6
/avatars/2ff141244521fd11bdd8eb147d5db127.svg
Charlie Chen
user
CharlieChen
67e5267538e4d1444c8bb99b
Charline Le Lan
67e5267538e4d1444c8bb99c
Christopher A. Choquette-Choo
admin_assigned
2025-03-28T16:33:45.001Z
649c5307e89a7650158f4965
/avatars/3c71b501db27c86ed686ae26ad1391cd.svg
Christopher A. Choquette-Choo
user
cchoquette
67e5267538e4d1444c8bb99d
CJ Carey
67e5267538e4d1444c8bb99e
Cormac Brick
67e5267538e4d1444c8bb99f
Daniel Deutsch
67e5267538e4d1444c8bb9a0
Danielle Eisenbud
67e5267538e4d1444c8bb9a1
Dee Cattle
67e5267538e4d1444c8bb9a2
Derek Cheng
67e5267538e4d1444c8bb9a3
Dimitris Paparas
67e5267538e4d1444c8bb9a4
Divyashree Shivakumar Sreepathihalli
admin_assigned
2025-03-28T16:33:55.270Z
667a0a4f1e2351f444ffda53
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/9jC7Sxh3sv712-Rszn-9F.jpeg
Divyashree Shivakumar Sreepathihalli
user
Divyasreepat
67e5267538e4d1444c8bb9a5
Doug Reid
67e5267538e4d1444c8bb9a6
Dustin Tran
67e5267538e4d1444c8bb9a7
Dustin Zelle
67e5267538e4d1444c8bb9a8
Eric Noland
67e5267538e4d1444c8bb9a9
Erwin Huizenga
67e5267538e4d1444c8bb9aa
Eugene Kharitonov
admin_assigned
2025-03-28T16:36:23.292Z
64509292fc5e5ff919b4a97d
https://cdn-avatars.huggingface.co/v1/production/uploads/64509292fc5e5ff919b4a97d/OOn5EaCXw39ePmgKptOuT.png
Eugene Kharitonov
user
ekharitonov
67e5267538e4d1444c8bb9ab
Frederick Liu
67e5267538e4d1444c8bb9ac
Gagik Amirkhanyan
admin_assigned
2025-03-28T16:36:14.818Z
6454043913dba495b2dff974
/avatars/8ff65ea746807205c243b8e1ef16fde8.svg
Gagik Amirkhanyan
user
Gagika
67e5267538e4d1444c8bb9ad
Glenn Cameron
67e5267538e4d1444c8bb9ae
Hadi Hashemi
67e5267538e4d1444c8bb9af
Hanna Klimczak-Plucińska
67e5267538e4d1444c8bb9b0
Harman Singh
67e5267538e4d1444c8bb9b1
Harsh Mehta
67e5267538e4d1444c8bb9b2
Harshal Tushar Lehri
67e5267538e4d1444c8bb9b3
Hussein Hazimeh
admin_assigned
2025-03-28T16:34:31.015Z
678bbba070768ae96ef1fb29
/avatars/c4d8be0db958d7975172906fc1969aec.svg
Hussein Hazimeh
user
HusseinHzmh
67e5267538e4d1444c8bb9b4
Ian Ballantyne
67e5267538e4d1444c8bb9b5
Idan Szpektor
67e5267538e4d1444c8bb9b6
Ivan Nardini
67e5267538e4d1444c8bb9b7
Jean Pouget-Abadie
67e5267538e4d1444c8bb9b8
Jetha Chan
admin_assigned
2025-03-28T16:37:33.012Z
6670e8966d34fe6262885205
/avatars/f5f7845a54c2bf8e418989f113272578.svg
Jetha Chan
user
jethac
67e5267538e4d1444c8bb9b9
Joe Stanton
67e5267538e4d1444c8bb9ba
John Wieting
admin_assigned
2025-03-28T16:34:06.809Z
623d449d38540758f05ccfc4
/avatars/d79808850b1a2ef5fe23e08eb6f22d57.svg
John Wieting
user
jwieting
67e5267538e4d1444c8bb9bb
Jonathan Lai
67e5267538e4d1444c8bb9bc
Jordi Orbay
67e5267538e4d1444c8bb9bd
Joseph Fernandez
67e5267538e4d1444c8bb9be
Josh Newlan
admin_assigned
2025-03-28T16:37:23.951Z
631984ab1328b6caf9fececa
/avatars/a31d31d7820103e45ac430dba67619af.svg
Josh Newlan
user
joshnewlan
67e5267538e4d1444c8bb9bf
Ju-yeong Ji
admin_assigned
2025-03-28T16:38:39.549Z
6439c159d45e8db3e28986ea
https://cdn-avatars.huggingface.co/v1/production/uploads/6439c159d45e8db3e28986ea/YoSzP_3DUmev9_v5tXWds.jpeg
Juyeong Ji
user
bebechien
67e5267538e4d1444c8bb9c0
Jyotinder Singh
67e5267538e4d1444c8bb9c1
Kat Black
67e5267538e4d1444c8bb9c2
Kathy Yu
67e5267538e4d1444c8bb9c3
Kevin Hui
67e5267538e4d1444c8bb9c4
Kiran Vodrahalli
claimed_verified
2025-05-28T09:01:52.968Z
6835eea6b34ab1d58e6b8133
/avatars/eb013082aba7cd8688e16ea56d4a213c.svg
Kiran Vodrahalli
user
kiranvodrahalli
67e5267538e4d1444c8bb9c5
Klaus Greff
67e5267538e4d1444c8bb9c6
Linhai Qiu
admin_assigned
2025-03-28T16:40:01.734Z
65ab594427f7d6995aad58f2
/avatars/c417c4b99adfb554714c68e1d25f3a56.svg
qiulinhai
user
qiu2024
67e5267538e4d1444c8bb9c7
Marcella Valentine
67e5267538e4d1444c8bb9c8
Marina Coelho
67e5267538e4d1444c8bb9c9
Marvin Ritter
67e5267538e4d1444c8bb9ca
Matt Hoffman
67e5267538e4d1444c8bb9cb
Matthew Watson
67e5267538e4d1444c8bb9cc
Mayank Chaturvedi
67e5267538e4d1444c8bb9cd
Michael Moynihan
admin_assigned
2025-03-28T16:33:31.912Z
64538c820b98fb78611255ac
/avatars/24abedf9a95c2a9c449950f20f8c22c3.svg
Michael Moynihan
user
michael-boonan
67e5267538e4d1444c8bb9ce
Min Ma
67e5267538e4d1444c8bb9cf
Nabila Babar
admin_assigned
2025-03-28T16:40:11.800Z
6779de0315ed5fab589d6fb6
/avatars/c2c86ee084ab190f80d85ec7607e961c.svg
Nabila Babar
user
Nababar
67e5267538e4d1444c8bb9d0
Natasha Noy
admin_assigned
2025-03-28T16:39:08.225Z
6402abc5fa1acad60069439c
/avatars/e24e95fabddf1423eb649c3e37c29945.svg
Natasha Noy
user
NatashaFN
67e5267538e4d1444c8bb9d1
Nathan Byrd
67e5267538e4d1444c8bb9d2
Nick Roy
67e5267538e4d1444c8bb9d3
Nikola Momchev
67e5267538e4d1444c8bb9d4
Nilay Chauhan
67e5267538e4d1444c8bb9d5
Noveen Sachdeva
67e5267538e4d1444c8bb9d6
Oskar Bunyan
67e5267538e4d1444c8bb9d7
Pankil Botarda
67e5267538e4d1444c8bb9d8
Paul Caron
67e5267538e4d1444c8bb9d9
Paul Kishan Rubenstein
67e5267538e4d1444c8bb9da
Phil Culliton
admin_assigned
2025-03-28T16:36:48.352Z
627a9aa0f967ef475abe8037
/avatars/47cf94d6c015577d0a860b617c9ea2b8.svg
Phil Culliton
user
PhilCulliton
67e5267538e4d1444c8bb9db
Philipp Schmid
admin_assigned
2025-03-28T16:36:36.617Z
5ff5d596f244529b3ec0fb89
https://cdn-avatars.huggingface.co/v1/production/uploads/1624629516652-5ff5d596f244529b3ec0fb89.png
Philipp Schmid
user
philschmid
67e5267538e4d1444c8bb9dc
Pier Giuseppe Sessa
67e5267538e4d1444c8bb9dd
Pingmei Xu
admin_assigned
2025-03-28T16:37:13.584Z
64781958950ad3b14dde3bfe
/avatars/78d8fb5a7c6a206f99e1cb4e68313187.svg
Pingmei Xu
user
pingmeix
67e5267538e4d1444c8bb9de
Piotr Stanczyk
admin_assigned
2025-03-28T16:38:48.128Z
67e2d0d135d3c9a9d93b6a70
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/HEMWc9pqX6qWO1pVeMvLg.png
Stanczyk
user
PiotrStanczyk
67e5267538e4d1444c8bb9df
Pouya Tafti
67e5267538e4d1444c8bb9e0
Rakesh Shivanna
67e5267538e4d1444c8bb9e1
Renjie Wu
67e5267538e4d1444c8bb9e2
Renke Pan
67e5267538e4d1444c8bb9e3
Reza Rokni
67e5267538e4d1444c8bb9e4
Rob Willoughby
admin_assigned
2025-03-28T16:38:05.300Z
669583d60e08a6505b5db0a3
/avatars/c8afae9611e1b7ac60c746d943b5734a.svg
Rob Willoughby
user
willougr
67e5267538e4d1444c8bb9e5
Rohith Vallu
67e5267538e4d1444c8bb9e6
Ryan Mullins
claimed_verified
2025-03-28T08:38:16.921Z
65a402a53522df7a27125823
https://cdn-avatars.huggingface.co/v1/production/uploads/65a402a53522df7a27125823/tVR6lZlKmjPqi7dTZBGoS.jpeg
Ryan Mullins
user
RyanMullins
67e5267538e4d1444c8bb9e7
Sammy Jerome
67e5267538e4d1444c8bb9e8
Sara Smoot
67e5267538e4d1444c8bb9e9
Sertan Girgin
67e5267538e4d1444c8bb9ea
Shariq Iqbal
67e5267538e4d1444c8bb9eb
Shashir Reddy
67e5267538e4d1444c8bb9ec
Shruti Sheth
67e5267538e4d1444c8bb9ed
Siim Põder
67e5267538e4d1444c8bb9ee
Sijal Bhatnagar
67e5267538e4d1444c8bb9ef
Sindhu Raghuram Panyam
admin_assigned
2025-03-28T16:39:16.300Z
66328782bc4ab4b4505365d6
/avatars/5a469c24dc252b4b6e5710b7385724ed.svg
Sindhu Raghuram Panyam
user
SindhuRaghuram97
67e5267538e4d1444c8bb9f0
Sivan Eiger
67e5267538e4d1444c8bb9f1
Susan Zhang
67e5267538e4d1444c8bb9f2
Tianqi Liu
67e5267538e4d1444c8bb9f3
Trevor Yacovone
67e5267538e4d1444c8bb9f4
Tyler Liechty
67e5267538e4d1444c8bb9f5
Uday Kalra
67e5267538e4d1444c8bb9f6
Utku Evci
67e5267538e4d1444c8bb9f7
Vedant Misra
67e5267538e4d1444c8bb9f8
Vincent Roseberry
admin_assigned
2025-03-28T16:37:41.167Z
624de3642604e14616ad0310
/avatars/db33f1d6b0985e0b37b5d4e30aff27f5.svg
Vincent Roseberry
user
rosbo
67e5267538e4d1444c8bb9f9
Vlad Feinberg
67e5267538e4d1444c8bb9fa
Vlad Kolesnikov
67e5267538e4d1444c8bb9fb
Woohyun Han
67e5267538e4d1444c8bb9fc
Woosuk Kwon
67e5267538e4d1444c8bb9fd
Xi Chen
67e5267538e4d1444c8bb9fe
Yinlam Chow
67e5267538e4d1444c8bb9ff
Yuvein Zhu
67e5267538e4d1444c8bba00
Zichuan Wei
67e5267538e4d1444c8bba01
Zoltan Egyed
67e5267538e4d1444c8bba02
Victor Cotruta
67e5267538e4d1444c8bba03
Minh Giang
67e5267538e4d1444c8bba04
Phoebe Kirk
67e5267538e4d1444c8bba05
Anand Rao
67e5267538e4d1444c8bba06
Kat Black
67e5267538e4d1444c8bba07
Nabila Babar
67e5267538e4d1444c8bba08
Jessica Lo
67e5267538e4d1444c8bba09
Erica Moreira
67e5267538e4d1444c8bba0a
Luiz Gustavo Martins
67e5267538e4d1444c8bba0b
Omar Sanseviero
claimed_verified
2025-03-28T16:22:42.660Z
6032802e1f993496bc14d9e3
https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png
Omar Sanseviero
user
osanseviero
67e5267538e4d1444c8bba0c
Lucas Gonzalez
67e5267538e4d1444c8bba0d
Zach Gleicher
67e5267538e4d1444c8bba0e
Tris Warkentin
67e5267538e4d1444c8bba0f
Vahab Mirrokni
67e5267538e4d1444c8bba10
Evan Senter
admin_assigned
2025-03-28T16:34:59.174Z
62a47ec0fdd7a13aac264904
/avatars/f2ff23645203ed2399cfe8b447af437a.svg
Evan Senter
user
evansenter
67e5267538e4d1444c8bba11
Eli Collins
67e5267538e4d1444c8bba12
Joelle Barral
67e5267538e4d1444c8bba13
Zoubin Ghahramani
67e5267538e4d1444c8bba14
Raia Hadsell
67e5267538e4d1444c8bba15
Yossi Matias
67e5267538e4d1444c8bba16
D. Sculley
67e5267538e4d1444c8bba17
Slav Petrov
67e5267538e4d1444c8bba18
Noah Fiedel
67e5267538e4d1444c8bba19
Noam Shazeer
67e5267538e4d1444c8bba1a
Oriol Vinyals
67e5267538e4d1444c8bba1b
Jeff Dean
67e5267538e4d1444c8bba1c
Demis Hassabis
67e5267538e4d1444c8bba1d
Koray Kavukcuoglu
67e5267538e4d1444c8bba1e
Clement Farabet
67e5267538e4d1444c8bba1f
Elena Buchatskaya
admin_assigned
2025-03-28T16:39:23.676Z
65da0ecba37e8a543ab732ca
/avatars/90889950d8c5e0089ada6b3707104a87.svg
Elena Buchatskaya
user
ebuchatskaya
67e5267538e4d1444c8bba20
Jean-Baptiste Alayrac
67e5267538e4d1444c8bba21
Rohan Anil
67e5267538e4d1444c8bba22
Dmitry
67e5267538e4d1444c8bba23
Lepikhin
67e5267538e4d1444c8bba24
Sebastian Borgeaud
67e5267538e4d1444c8bba25
Olivier Bachem
67e5267538e4d1444c8bba26
Armand Joulin
67e5267538e4d1444c8bba27
Alek Andreev
admin_assigned
2025-03-28T16:32:10.502Z
65d667858ac5cf899e3b66c4
/avatars/340468ee8cd9125eceae23a9b8e536a5.svg
Alek Andreev
user
alekandreev
67e5267538e4d1444c8bba28
Cassidy Hardin
admin_assigned
2025-03-28T16:32:00.997Z
663a5ec5a4152b77b6cf96fd
/avatars/a51901b21d8670e26b2ac57fd482bb63.svg
Cassidy Hardin
user
casshardin
67e5267538e4d1444c8bba29
Robert Dadashi
admin_assigned
2025-03-28T16:31:53.064Z
65fc7c0da57b88a765aea493
/avatars/440f88769da1113d6158fe7e0514ead3.svg
Robert Dadashi
user
ddsh
67e5267538e4d1444c8bba2a
Léonard Hussenot
claimed_verified
2025-03-28T14:22:28.133Z
667a7b2e1764931987803b46
/avatars/da2ee162f33c681e3d1c4d1e6d44dcbd.svg
Léonard Hussenot
user
leonardhussenot
Gemma 3 Technical Report
We introduce Gemma 3, a multimodal addition to the Gemma family of
lightweight open models, ranging in scale from 1 to 27 billion parameters. This
version introduces vision understanding abilities, a wider coverage of
languages and longer context - at least 128K tokens. We also change the
architecture of the model to reduce the KV-cache memory that tends to explode
with long context. This is achieved by increasing the ratio of local to global
attention layers, and keeping the span on local attention short. The Gemma 3
models are trained with distillation and achieve superior performance to Gemma
2 for both pre-trained and instruction finetuned versions. In particular, our
novel post-training recipe significantly improves the math, chat,
instruction-following and multilingual abilities, making Gemma3-4B-IT
competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro
across benchmarks. We release all our models to the community.
Gemma 3 introduces vision capabilities, broader language coverage, and extended context length, featuring an optimized architecture and post-training enhancements to outperform previous versions.
multimodal models vision understanding long context KV-cache memory local attention global attention model distillation pre-trained models instruction finetuned post-training recipe math ability chat ability instruction-following multilingual abilities