title: Embodied Red Teaming for Auditing Robotic Foundation Models
summary: Language-conditioned robot models have the potential to enable robots to
  perform a wide range of tasks based on natural language instructions. However,
  assessing their safety and effectiveness remains challenging because it is
  difficult to test all the different ways a single task can be phrased. Current
  benchmarks have two key limitations: they rely on a limited set of
  human-generated instructions, missing many challenging cases, and focus only on
  task performance without assessing safety, such as avoiding damage. To address
  these gaps, we introduce Embodied Red Teaming (ERT), a new evaluation method
  that generates diverse and challenging instructions to test these models. ERT
  uses automated red teaming techniques with Vision Language Models (VLMs) to
  create contextually grounded, difficult instructions. Experimental results show
  that state-of-the-art language-conditioned robot models fail or behave unsafely
  on ERT-generated instructions, underscoring the shortcomings of current
  benchmarks in evaluating real-world performance and safety. Code and videos are
  available at: https://s-karnik.github.io/embodied-red-team-project-page.
paper_authors._id: 67ab837b02329ca8f809ceae
paper_authors.name: Sathwik Karnik
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-12T09:16:48.921Z
paper_authors._id: 67ab79ead3f12c4887867022
paper_authors.avatarUrl: /avatars/d8e1dd4afabd4c2d04a73712a610cea6.svg
paper_authors.fullname: Sathwik Karnik
paper_authors.type: user
paper_authors.user: S-Karnik
paper_authors._id: 67ab837b02329ca8f809ceaf
paper_authors.name: Zhang-Wei Hong
paper_authors._id: 67ab837b02329ca8f809ceb0
paper_authors.name: Nishant Abhangi
paper_authors._id: 67ab837b02329ca8f809ceb1
paper_authors.name: Yen-Chen Lin
paper_authors._id: 67ab837b02329ca8f809ceb2
paper_authors.name: Tsun-Hsuan Wang
paper_authors._id: 67ab837b02329ca8f809ceb3
paper_authors.name: Christophe Dupuy
paper_authors._id: 67ab837b02329ca8f809ceb4
paper_authors.name: Rahul Gupta
paper_authors._id: 67ab837b02329ca8f809ceb5
paper_authors.name: Pulkit Agrawal
paper_title: Embodied Red Teaming for Auditing Robotic Foundation Models
paper_summary: Language-conditioned robot models have the potential to enable robots to
  perform a wide range of tasks based on natural language instructions. However,
  assessing their safety and effectiveness remains challenging because it is
  difficult to test all the different ways a single task can be phrased. Current
  benchmarks have two key limitations: they rely on a limited set of
  human-generated instructions, missing many challenging cases, and focus only on
  task performance without assessing safety, such as avoiding damage. To address
  these gaps, we introduce Embodied Red Teaming (ERT), a new evaluation method
  that generates diverse and challenging instructions to test these models. ERT
  uses automated red teaming techniques with Vision Language Models (VLMs) to
  create contextually grounded, difficult instructions. Experimental results show
  that state-of-the-art language-conditioned robot models fail or behave unsafely
  on ERT-generated instructions, underscoring the shortcomings of current
  benchmarks in evaluating real-world performance and safety. Code and videos are
  available at: https://s-karnik.github.io/embodied-red-team-project-page.
paper_ai_summary: Embodied Red Teaming uses automated red teaming with Vision Language Models to generate diverse, challenging instructions for evaluating the safety and effectiveness of language-conditioned robot models.
paper_ai_keywords: Vision Language Models embodied red teaming contextually grounded state-of-the-art language-conditioned robot models