title: Ruby Teaming: Improving Quality Diversity Search with Memory for
    Automated Red Teaming
summary: We propose Ruby Teaming, a method that improves on Rainbow Teaming by
  including a memory cache as its third dimension. The memory dimension provides
  cues to the mutator to yield better-quality prompts, both in terms of attack
  success rate (ASR) and quality diversity. The prompt archive generated by Ruby
  Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of
  quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on
  Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI),
  respectively.
paper_authors._id: 667987754e7665bb49695f52
paper_authors.name: Vernon Toh Yan Han
paper_authors._id: 667987754e7665bb49695f53
paper_authors.name: Rishabh Bhardwaj
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-19T07:08:26.864Z
paper_authors._id: 5f278507e923d665e616271b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f278507e923d665e616271b/tWFuswXOTXtvMdL8zSrr_.png
paper_authors.fullname: Rishabh Bhardwaj
paper_authors.type: user
paper_authors.user: RishabhBhardwaj
paper_authors._id: 667987754e7665bb49695f54
paper_authors.name: Soujanya Poria
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-22T19:52:34.994Z
paper_authors._id: 626b626405fe1cb65725aca1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/aa-Lata46I3fXOmMetvXH.jpeg
paper_authors.fullname: Soujanya Poria
paper_authors.type: user
paper_authors.user: soujanyaporia
paper_title: Ruby Teaming: Improving Quality Diversity Search with Memory for
    Automated Red Teaming
paper_summary: We propose Ruby Teaming, a method that improves on Rainbow Teaming by
  including a memory cache as its third dimension. The memory dimension provides
  cues to the mutator to yield better-quality prompts, both in terms of attack
  success rate (ASR) and quality diversity. The prompt archive generated by Ruby
  Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of
  quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on
  Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI),
  respectively.
paper_ai_keywords: memory cache mutator prompts attack success rate (ASR) quality diversity prompt archive Shannon's Evenness Index (SEI) Simpson's Diversity Index (SDI)