title: Teaching Models to Balance Resisting and Accepting Persuasion
summary: Large language models (LLMs) are susceptible to persuasion, which can pose
  risks when models are faced with an adversarial interlocutor. We take a first
  step towards defending models against persuasion while also arguing that
  defense against adversarial (i.e. negative) persuasion is only half of the
  equation: models should also be able to accept beneficial (i.e. positive)
  persuasion to improve their answers. We show that optimizing models for only
  one side results in poor performance on the other. In order to balance positive
  and negative persuasion, we introduce Persuasion-Balanced Training (or PBT),
  which leverages multi-agent recursive dialogue trees to create data and trains
  models via preference optimization to accept persuasion when appropriate. PBT
  consistently improves resistance to misinformation and resilience to being
  challenged while also resulting in the best overall performance on holistic
  data containing both positive and negative persuasion. Crucially, we show that
  PBT models are better teammates in multi-agent debates. We find that without
  PBT, pairs of stronger and weaker models have unstable performance, with the
  order in which the models present their answers determining whether the team
  obtains the stronger or weaker model's performance. PBT leads to better and
  more stable results and less order dependence, with the stronger model
  consistently pulling the weaker one up.
paper_authors._id: 67165dcc03d83371d444d2c2
paper_authors.name: Elias Stengel-Eskin
paper_authors._id: 67165dcc03d83371d444d2c3
paper_authors.name: Peter Hase
paper_authors._id: 67165dcc03d83371d444d2c4
paper_authors.name: Mohit Bansal
paper_title: Teaching Models to Balance Resisting and Accepting Persuasion
paper_summary: Large language models (LLMs) are susceptible to persuasion, which can pose
  risks when models are faced with an adversarial interlocutor. We take a first
  step towards defending models against persuasion while also arguing that
  defense against adversarial (i.e. negative) persuasion is only half of the
  equation: models should also be able to accept beneficial (i.e. positive)
  persuasion to improve their answers. We show that optimizing models for only
  one side results in poor performance on the other. In order to balance positive
  and negative persuasion, we introduce Persuasion-Balanced Training (or PBT),
  which leverages multi-agent recursive dialogue trees to create data and trains
  models via preference optimization to accept persuasion when appropriate. PBT
  consistently improves resistance to misinformation and resilience to being
  challenged while also resulting in the best overall performance on holistic
  data containing both positive and negative persuasion. Crucially, we show that
  PBT models are better teammates in multi-agent debates. We find that without
  PBT, pairs of stronger and weaker models have unstable performance, with the
  order in which the models present their answers determining whether the team
  obtains the stronger or weaker model's performance. PBT leads to better and
  more stable results and less order dependence, with the stronger model
  consistently pulling the weaker one up.
paper_ai_summary: Persuasion-Balanced Training (PBT) improves LLMs' resistance to misinformation and resilience to challenges by optimizing them for both positive and negative persuasion, leading to better performance in multi-agent debates.
paper_ai_keywords: Large language models LLMs persuasion adversarial persuasion positive persuasion Persuasion-Balanced Training PBT multi-agent recursive dialogue trees preference optimization misinformation resilience multi-agent debates order dependence