title: Lessons from Defending Gemini Against Indirect Prompt Injections
summary: Gemini is increasingly used to perform tasks on behalf of users, where
  function-calling and tool-use capabilities enable the model to access user
  data. Some tools, however, require access to untrusted data introducing risk.
  Adversaries can embed malicious instructions in untrusted data which cause the
  model to deviate from the user's expectations and mishandle their data or
  permissions. In this report, we set out Google DeepMind's approach to
  evaluating the adversarial robustness of Gemini models and describe the main
  lessons learned from the process. We test how Gemini performs against a
  sophisticated adversary through an adversarial evaluation framework, which
  deploys a suite of adaptive attack techniques to run continuously against past,
  current, and future versions of Gemini. We describe how these ongoing
  evaluations directly help make Gemini more resilient against manipulation.
paper_authors._id: 682d7f009a06bc9f9106480b
paper_authors.name: Chongyang Shi
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-21T07:21:37.326Z
paper_authors._id: 65ef0b4b325d9aaef87eb33b
paper_authors.avatarUrl: /avatars/ec27dba9de7e74c1f6cdcacf5aa25528.svg
paper_authors.fullname: C Shi
paper_authors.type: user
paper_authors.user: chongyangs
paper_authors._id: 682d7f009a06bc9f9106480c
paper_authors.name: Sharon Lin
paper_authors._id: 682d7f009a06bc9f9106480d
paper_authors.name: Shuang Song
paper_authors._id: 682d7f009a06bc9f9106480e
paper_authors.name: Jamie Hayes
paper_authors._id: 682d7f009a06bc9f9106480f
paper_authors.name: Ilia Shumailov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:31:09.839Z
paper_authors._id: 6475c2794766357252e69e9f
paper_authors.avatarUrl: /avatars/757ed789423113369868e972f21ce559.svg
paper_authors.fullname: i
paper_authors.type: user
paper_authors.user: iliashum
paper_authors._id: 682d7f009a06bc9f91064810
paper_authors.name: Itay Yona
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:31:16.965Z
paper_authors._id: 62697edaa6a7bba9e46ae4ad
paper_authors.avatarUrl: /avatars/3009bf769a09d946447a0e8c833a04f3.svg
paper_authors.fullname: Itay Yona
paper_authors.type: user
paper_authors.user: tux
paper_authors._id: 682d7f009a06bc9f91064811
paper_authors.name: Juliette Pluto
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:31:22.677Z
paper_authors._id: 657d17d43480ce8aae628d40
paper_authors.avatarUrl: /avatars/7f8cd9953f82e5a7ad5db1f82cfdb8f1.svg
paper_authors.fullname: juliette pluto
paper_authors.type: user
paper_authors.user: julsh
paper_authors._id: 682d7f009a06bc9f91064812
paper_authors.name: Aneesh Pappu
paper_authors._id: 682d7f009a06bc9f91064813
paper_authors.name: Christopher A. Choquette-Choo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:31:33.340Z
paper_authors._id: 649c5307e89a7650158f4965
paper_authors.avatarUrl: /avatars/3c71b501db27c86ed686ae26ad1391cd.svg
paper_authors.fullname: Christopher A. Choquette-Choo
paper_authors.type: user
paper_authors.user: cchoquette
paper_authors._id: 682d7f009a06bc9f91064814
paper_authors.name: Milad Nasr
paper_authors._id: 682d7f009a06bc9f91064815
paper_authors.name: Chawin Sitawarin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:31:58.593Z
paper_authors._id: 623e3ebef3be5bae969680c0
paper_authors.avatarUrl: /avatars/590c264bf9964225bd67f61a3a5d53ea.svg
paper_authors.fullname: Chawin Sitawarin
paper_authors.type: user
paper_authors.user: chawins
paper_authors._id: 682d7f009a06bc9f91064816
paper_authors.name: Gena Gibson
paper_authors._id: 682d7f009a06bc9f91064817
paper_authors.name: Andreas Terzis
paper_authors._id: 682d7f009a06bc9f91064818
paper_authors.name: John "Four" Flynn
paper_title: Lessons from Defending Gemini Against Indirect Prompt Injections
paper_summary: Gemini is increasingly used to perform tasks on behalf of users, where
  function-calling and tool-use capabilities enable the model to access user
  data. Some tools, however, require access to untrusted data introducing risk.
  Adversaries can embed malicious instructions in untrusted data which cause the
  model to deviate from the user's expectations and mishandle their data or
  permissions. In this report, we set out Google DeepMind's approach to
  evaluating the adversarial robustness of Gemini models and describe the main
  lessons learned from the process. We test how Gemini performs against a
  sophisticated adversary through an adversarial evaluation framework, which
  deploys a suite of adaptive attack techniques to run continuously against past,
  current, and future versions of Gemini. We describe how these ongoing
  evaluations directly help make Gemini more resilient against manipulation.
paper_ai_summary: Google DeepMind evaluates the adversarial robustness of Gemini through continuous testing with adaptive attack techniques to enhance its resilience.
paper_ai_keywords: adversarial robustness model evaluation adaptive attack techniques