title: Gemini Robotics: Bringing AI into the Physical World
summary: Recent advancements in large multimodal models have led to the emergence of
  remarkable generalist capabilities in digital domains, yet their translation to
  physical agents such as robots remains a significant challenge. This report
  introduces a new family of AI models purposefully designed for robotics and
  built upon the foundation of Gemini 2.0. We present Gemini Robotics, an
  advanced Vision-Language-Action (VLA) generalist model capable of directly
  controlling robots. Gemini Robotics executes smooth and reactive movements to
  tackle a wide range of complex manipulation tasks while also being robust to
  variations in object types and positions, handling unseen environments as well
  as following diverse, open vocabulary instructions. We show that with
  additional fine-tuning, Gemini Robotics can be specialized to new capabilities
  including solving long-horizon, highly dexterous tasks, learning new
  short-horizon tasks from as few as 100 demonstrations and adapting to
  completely novel robot embodiments. This is made possible because Gemini
  Robotics builds on top of the Gemini Robotics-ER model, the second model we
  introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends
  Gemini's multimodal reasoning capabilities into the physical world, with
  enhanced spatial and temporal understanding. This enables capabilities relevant
  to robotics including object detection, pointing, trajectory and grasp
  prediction, as well as multi-view correspondence and 3D bounding box
  predictions. We show how this novel combination can support a variety of
  robotics applications. We also discuss and address important safety
  considerations related to this new class of robotics foundation models. The
  Gemini Robotics family marks a substantial step towards developing
  general-purpose robots that realizes AI's potential in the physical world.
paper_authors._id: 67e4b288fa81c69f446da710
paper_authors.name: Gemini Robotics Team
paper_authors._id: 67e4b288fa81c69f446da711
paper_authors.name: Saminda Abeyruwan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:58:16.630Z
paper_authors._id: 61fd7ac3fbafe89f48101d83
paper_authors.avatarUrl: /avatars/cb2c86a04574498e71d6c447c2b289c1.svg
paper_authors.fullname: Saminda Abeyruwan
paper_authors.type: user
paper_authors.user: saminda
paper_authors._id: 67e4b288fa81c69f446da712
paper_authors.name: Joshua Ainslie
paper_authors._id: 67e4b288fa81c69f446da713
paper_authors.name: Jean-Baptiste Alayrac
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:58:27.846Z
paper_authors._id: 6253ee39e4e98393660b5c35
paper_authors.avatarUrl: /avatars/9c032f6a0729bfe5c16b3affe190834d.svg
paper_authors.fullname: Jean-Baptiste Alayrac
paper_authors.type: user
paper_authors.user: jalayrac
paper_authors._id: 67e4b288fa81c69f446da714
paper_authors.name: Montserrat Gonzalez Arenas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:58:33.087Z
paper_authors._id: 63f47a88121f9894707465ed
paper_authors.avatarUrl: /avatars/d85d409d19068aea02a2532b587dd1ef.svg
paper_authors.fullname: Montserrat Gonzalez Arenas
paper_authors.type: user
paper_authors.user: montse90
paper_authors._id: 67e4b288fa81c69f446da715
paper_authors.name: Travis Armstrong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:58:39.411Z
paper_authors._id: 66140283cf3fef4fa812e92f
paper_authors.avatarUrl: /avatars/033586adc3931d6c85bf9e84220992b4.svg
paper_authors.fullname: Travis Armstrong
paper_authors.type: user
paper_authors.user: TravisAStrong
paper_authors._id: 67e4b288fa81c69f446da716
paper_authors.name: Ashwin Balakrishna
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:58:46.130Z
paper_authors._id: 653ad36e5f1703225b266b7b
paper_authors.avatarUrl: /avatars/170e6b54a9859c7fca0289a09654c47f.svg
paper_authors.fullname: Ashwin Balakrishna
paper_authors.type: user
paper_authors.user: abalakrishna123
paper_authors._id: 67e4b288fa81c69f446da717
paper_authors.name: Robert Baruch
paper_authors._id: 67e4b288fa81c69f446da718
paper_authors.name: Maria Bauza
paper_authors._id: 67e4b288fa81c69f446da719
paper_authors.name: Michiel Blokzijl
paper_authors._id: 67e4b288fa81c69f446da71a
paper_authors.name: Steven Bohez
paper_authors._id: 67e4b288fa81c69f446da71b
paper_authors.name: Konstantinos Bousmalis
paper_authors._id: 67e4b288fa81c69f446da71c
paper_authors.name: Anthony Brohan
paper_authors._id: 67e4b288fa81c69f446da71d
paper_authors.name: Thomas Buschmann
paper_authors._id: 67e4b288fa81c69f446da71e
paper_authors.name: Arunkumar Byravan
paper_authors._id: 67e4b288fa81c69f446da71f
paper_authors.name: Serkan Cabi
paper_authors._id: 67e4b288fa81c69f446da720
paper_authors.name: Ken Caluwaerts
paper_authors._id: 67e4b288fa81c69f446da721
paper_authors.name: Federico Casarini
paper_authors._id: 67e4b288fa81c69f446da722
paper_authors.name: Oscar Chang
paper_authors._id: 67e4b288fa81c69f446da723
paper_authors.name: Jose Enrique Chen
paper_authors._id: 67e4b288fa81c69f446da724
paper_authors.name: Xi Chen
paper_authors._id: 67e4b288fa81c69f446da725
paper_authors.name: Hao-Tien Lewis Chiang
paper_authors._id: 67e4b288fa81c69f446da726
paper_authors.name: Krzysztof Choromanski
paper_authors._id: 67e4b288fa81c69f446da727
paper_authors.name: David D'Ambrosio
paper_authors._id: 67e4b288fa81c69f446da728
paper_authors.name: Sudeep Dasari
paper_authors._id: 67e4b288fa81c69f446da729
paper_authors.name: Todor Davchev
paper_authors._id: 67e4b288fa81c69f446da72a
paper_authors.name: Coline Devin
paper_authors._id: 67e4b288fa81c69f446da72b
paper_authors.name: Norman Di Palo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:09.232Z
paper_authors._id: 62dac377f014388f908974f4
paper_authors.avatarUrl: /avatars/39bf0ca206441575a45f577060cdd8bc.svg
paper_authors.fullname: Norman Di Palo
paper_authors.type: user
paper_authors.user: normandipalo
paper_authors._id: 67e4b288fa81c69f446da72c
paper_authors.name: Tianli Ding
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:18.146Z
paper_authors._id: 64d89da3bab152b24713108e
paper_authors.avatarUrl: /avatars/22e10f9a13b0d18b3a3b1f5281c7124d.svg
paper_authors.fullname: Tianli Ding
paper_authors.type: user
paper_authors.user: Tding
paper_authors._id: 67e4b288fa81c69f446da72d
paper_authors.name: Adil Dostmohamed
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:29.517Z
paper_authors._id: 63b7d36469b7bd7324f9f438
paper_authors.avatarUrl: /avatars/67b1864c378102b1cf2de571cce7bf9a.svg
paper_authors.fullname: Adil Dostmohamed
paper_authors.type: user
paper_authors.user: adild
paper_authors._id: 67e4b288fa81c69f446da72e
paper_authors.name: Danny Driess
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:37.044Z
paper_authors._id: 67225875b46c703941fa7967
paper_authors.avatarUrl: /avatars/7c89fbdd9a135210209bcd0cbfe7988a.svg
paper_authors.fullname: Danny Driess
paper_authors.type: user
paper_authors.user: dannydriess
paper_authors._id: 67e4b288fa81c69f446da72f
paper_authors.name: Yilun Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:43.648Z
paper_authors._id: 63c9bd445fdc575773c732fe
paper_authors.avatarUrl: /avatars/def472d1ab3fbf751225357c0932ae7e.svg
paper_authors.fullname: Yilun Du
paper_authors.type: user
paper_authors.user: yilundu
paper_authors._id: 67e4b288fa81c69f446da730
paper_authors.name: Debidatta Dwibedi
paper_authors._id: 67e4b288fa81c69f446da731
paper_authors.name: Michael Elabd
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:00:55.368Z
paper_authors._id: 66f6e9a737473469e871cae8
paper_authors.avatarUrl: /avatars/8c247380cee5d879aac204299963d3a7.svg
paper_authors.fullname: Michael Elabd
paper_authors.type: user
paper_authors.user: michaelelabd
paper_authors._id: 67e4b288fa81c69f446da732
paper_authors.name: Claudio Fantacci
paper_authors._id: 67e4b288fa81c69f446da733
paper_authors.name: Cody Fong
paper_authors._id: 67e4b288fa81c69f446da734
paper_authors.name: Erik Frey
paper_authors._id: 67e4b288fa81c69f446da735
paper_authors.name: Chuyuan Fu
paper_authors._id: 67e4b288fa81c69f446da736
paper_authors.name: Marissa Giustina
paper_authors._id: 67e4b288fa81c69f446da737
paper_authors.name: Keerthana Gopalakrishnan
paper_authors._id: 67e4b288fa81c69f446da738
paper_authors.name: Laura Graesser
paper_authors._id: 67e4b288fa81c69f446da739
paper_authors.name: Leonard Hasenclever
paper_authors._id: 67e4b288fa81c69f446da73a
paper_authors.name: Nicolas Heess
paper_authors._id: 67e4b288fa81c69f446da73b
paper_authors.name: Brandon Hernaez
paper_authors._id: 67e4b288fa81c69f446da73c
paper_authors.name: Alexander Herzog
paper_authors._id: 67e4b288fa81c69f446da73d
paper_authors.name: R. Alex Hofer
paper_authors._id: 67e4b288fa81c69f446da73e
paper_authors.name: Jan Humplik
paper_authors._id: 67e4b288fa81c69f446da73f
paper_authors.name: Atil Iscen
paper_authors._id: 67e4b288fa81c69f446da740
paper_authors.name: Mithun George Jacob
paper_authors._id: 67e4b288fa81c69f446da741
paper_authors.name: Deepali Jain
paper_authors._id: 67e4b288fa81c69f446da742
paper_authors.name: Ryan Julian
paper_authors._id: 67e4b288fa81c69f446da743
paper_authors.name: Dmitry Kalashnikov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:01:27.098Z
paper_authors._id: 64b7bf04a5018e3c7ca2ecda
paper_authors.avatarUrl: /avatars/4a434c344f68f2915c6e823262e62946.svg
paper_authors.fullname: Dmitry Kalashnikov
paper_authors.type: user
paper_authors.user: dmitry-kalashnikov
paper_authors._id: 67e4b288fa81c69f446da744
paper_authors.name: M. Emre Karagozler
paper_authors._id: 67e4b288fa81c69f446da745
paper_authors.name: Stefani Karp
paper_authors._id: 67e4b288fa81c69f446da746
paper_authors.name: Chase Kew
paper_authors._id: 67e4b288fa81c69f446da747
paper_authors.name: Jerad Kirkland
paper_authors._id: 67e4b288fa81c69f446da748
paper_authors.name: Sean Kirmani
paper_authors._id: 67e4b288fa81c69f446da749
paper_authors.name: Yuheng Kuang
paper_authors._id: 67e4b288fa81c69f446da74a
paper_authors.name: Thomas Lampe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:01:36.252Z
paper_authors._id: 631fa7e9124782a19efd20f2
paper_authors.avatarUrl: /avatars/456ef70cdb2a6a1f79a078746e96034a.svg
paper_authors.fullname: Thomas Lampe
paper_authors.type: user
paper_authors.user: tlampe
paper_authors._id: 67e4b288fa81c69f446da74b
paper_authors.name: Antoine Laurens
paper_authors._id: 67e4b288fa81c69f446da74c
paper_authors.name: Isabel Leal
paper_authors._id: 67e4b288fa81c69f446da74d
paper_authors.name: Alex X. Lee
paper_authors._id: 67e4b288fa81c69f446da74e
paper_authors.name: Tsang-Wei Edward Lee
paper_authors._id: 67e4b288fa81c69f446da74f
paper_authors.name: Jacky Liang
paper_authors._id: 67e4b288fa81c69f446da750
paper_authors.name: Yixin Lin
paper_authors._id: 67e4b288fa81c69f446da751
paper_authors.name: Sharath Maddineni
paper_authors._id: 67e4b288fa81c69f446da752
paper_authors.name: Anirudha Majumdar
paper_authors._id: 67e4b288fa81c69f446da753
paper_authors.name: Assaf Hurwitz Michaely
paper_authors._id: 67e4b288fa81c69f446da754
paper_authors.name: Robert Moreno
paper_authors._id: 67e4b288fa81c69f446da755
paper_authors.name: Michael Neunert
paper_authors._id: 67e4b288fa81c69f446da756
paper_authors.name: Francesco Nori
paper_authors._id: 67e4b288fa81c69f446da757
paper_authors.name: Carolina Parada
paper_authors._id: 67e4b288fa81c69f446da758
paper_authors.name: Emilio Parisotto
paper_authors._id: 67e4b288fa81c69f446da759
paper_authors.name: Peter Pastor
paper_authors._id: 67e4b288fa81c69f446da75a
paper_authors.name: Acorn Pooley
paper_authors._id: 67e4b288fa81c69f446da75b
paper_authors.name: Kanishka Rao
paper_authors._id: 67e4b288fa81c69f446da75c
paper_authors.name: Krista Reymann
paper_authors._id: 67e4b288fa81c69f446da75d
paper_authors.name: Dorsa Sadigh
paper_authors._id: 67e4b288fa81c69f446da75e
paper_authors.name: Stefano Saliceti
paper_authors._id: 67e4b288fa81c69f446da75f
paper_authors.name: Pannag Sanketi
paper_authors._id: 67e4b288fa81c69f446da760
paper_authors.name: Pierre Sermanet
paper_authors._id: 67e4b288fa81c69f446da761
paper_authors.name: Dhruv Shah
paper_authors._id: 67e4b288fa81c69f446da762
paper_authors.name: Mohit Sharma
paper_authors._id: 67e4b288fa81c69f446da763
paper_authors.name: Kathryn Shea
paper_authors._id: 67e4b288fa81c69f446da764
paper_authors.name: Charles Shu
paper_authors._id: 67e4b288fa81c69f446da765
paper_authors.name: Vikas Sindhwani
paper_authors._id: 67e4b288fa81c69f446da766
paper_authors.name: Sumeet Singh
paper_authors._id: 67e4b288fa81c69f446da767
paper_authors.name: Radu Soricut
paper_authors._id: 67e4b288fa81c69f446da768
paper_authors.name: Jost Tobias Springenberg
paper_authors._id: 67e4b288fa81c69f446da769
paper_authors.name: Rachel Sterneck
paper_authors._id: 67e4b288fa81c69f446da76a
paper_authors.name: Razvan Surdulescu
paper_authors._id: 67e4b288fa81c69f446da76b
paper_authors.name: Jie Tan
paper_authors._id: 67e4b288fa81c69f446da76c
paper_authors.name: Jonathan Tompson
paper_authors._id: 67e4b288fa81c69f446da76d
paper_authors.name: Vincent Vanhoucke
paper_authors._id: 67e4b288fa81c69f446da76e
paper_authors.name: Jake Varley
paper_authors._id: 67e4b288fa81c69f446da76f
paper_authors.name: Grace Vesom
paper_authors._id: 67e4b288fa81c69f446da770
paper_authors.name: Giulia Vezzani
paper_authors._id: 67e4b288fa81c69f446da771
paper_authors.name: Oriol Vinyals
paper_authors._id: 67e4b288fa81c69f446da772
paper_authors.name: Ayzaan Wahid
paper_authors._id: 67e4b288fa81c69f446da773
paper_authors.name: Stefan Welker
paper_authors._id: 67e4b288fa81c69f446da774
paper_authors.name: Paul Wohlhart
paper_authors._id: 67e4b288fa81c69f446da775
paper_authors.name: Fei Xia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-28T08:38:18.882Z
paper_authors._id: 62cfa9ee9b96f22525c36d47
paper_authors.avatarUrl: /avatars/8bfe3bf63f0609b3b6b7d7c5d7f6b3e9.svg
paper_authors.fullname: Fei Xia
paper_authors.type: user
paper_authors.user: fxia22
paper_authors._id: 67e4b288fa81c69f446da776
paper_authors.name: Ted Xiao
paper_authors._id: 67e4b288fa81c69f446da777
paper_authors.name: Annie Xie
paper_authors._id: 67e4b288fa81c69f446da778
paper_authors.name: Jinyu Xie
paper_authors._id: 67e4b288fa81c69f446da779
paper_authors.name: Peng Xu
paper_authors._id: 67e4b288fa81c69f446da77a
paper_authors.name: Sichun Xu
paper_authors._id: 67e4b288fa81c69f446da77b
paper_authors.name: Ying Xu
paper_authors._id: 67e4b288fa81c69f446da77c
paper_authors.name: Zhuo Xu
paper_authors._id: 67e4b288fa81c69f446da77d
paper_authors.name: Yuxiang Yang
paper_authors._id: 67e4b288fa81c69f446da77e
paper_authors.name: Rui Yao
paper_authors._id: 67e4b288fa81c69f446da77f
paper_authors.name: Sergey Yaroshenko
paper_authors._id: 67e4b288fa81c69f446da780
paper_authors.name: Wenhao Yu
paper_authors._id: 67e4b288fa81c69f446da781
paper_authors.name: Wentao Yuan
paper_authors._id: 67e4b288fa81c69f446da782
paper_authors.name: Jingwei Zhang
paper_authors._id: 67e4b288fa81c69f446da783
paper_authors.name: Tingnan Zhang
paper_authors._id: 67e4b288fa81c69f446da784
paper_authors.name: Allan Zhou
paper_authors._id: 67e4b288fa81c69f446da785
paper_authors.name: Yuxiang Zhou
paper_title: Gemini Robotics: Bringing AI into the Physical World
paper_summary: Recent advancements in large multimodal models have led to the emergence of
  remarkable generalist capabilities in digital domains, yet their translation to
  physical agents such as robots remains a significant challenge. This report
  introduces a new family of AI models purposefully designed for robotics and
  built upon the foundation of Gemini 2.0. We present Gemini Robotics, an
  advanced Vision-Language-Action (VLA) generalist model capable of directly
  controlling robots. Gemini Robotics executes smooth and reactive movements to
  tackle a wide range of complex manipulation tasks while also being robust to
  variations in object types and positions, handling unseen environments as well
  as following diverse, open vocabulary instructions. We show that with
  additional fine-tuning, Gemini Robotics can be specialized to new capabilities
  including solving long-horizon, highly dexterous tasks, learning new
  short-horizon tasks from as few as 100 demonstrations and adapting to
  completely novel robot embodiments. This is made possible because Gemini
  Robotics builds on top of the Gemini Robotics-ER model, the second model we
  introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends
  Gemini's multimodal reasoning capabilities into the physical world, with
  enhanced spatial and temporal understanding. This enables capabilities relevant
  to robotics including object detection, pointing, trajectory and grasp
  prediction, as well as multi-view correspondence and 3D bounding box
  predictions. We show how this novel combination can support a variety of
  robotics applications. We also discuss and address important safety
  considerations related to this new class of robotics foundation models. The
  Gemini Robotics family marks a substantial step towards developing
  general-purpose robots that realizes AI's potential in the physical world.
paper_ai_summary: Gemini Robotics, an advanced Vision-Language-Action model based on Gemini 2.0, controls robots with multimodal reasoning, handling complex tasks and adapting to new environments and instructions.
paper_ai_keywords: Vision-Language-Action multimodal reasoning spatial and temporal understanding object detection pointing trajectory prediction grasp prediction multi-view correspondence 3D bounding box predictions embodied reasoning