title: Surfacing Biases in Large Language Models using Contrastive Input
    Decoding
summary: Ensuring that large language models (LMs) are fair, robust and useful
  requires an understanding of how different modifications to their inputs impact
  the model's behaviour. In the context of open-text generation tasks, however,
  such an evaluation is not trivial. For example, when introducing a model with
  an input text and a perturbed, "contrastive" version of it, meaningful
  differences in the next-token predictions may not be revealed with standard
  decoding strategies. With this motivation in mind, we propose Contrastive Input
  Decoding (CID): a decoding algorithm to generate text given two inputs, where
  the generated text is likely given one input but unlikely given the other. In
  this way, the contrastive generations can highlight potentially subtle
  differences in how the LM output differs for the two inputs in a simple and
  interpretable manner. We use CID to highlight context-specific biases that are
  hard to detect with standard decoding strategies and quantify the effect of
  different input perturbations.
paper_authors._id: 64618f4f2815a070474da2ae
paper_authors.name: Gal Yona
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-16T12:23:15.486Z
paper_authors._id: 64637577c615cbc124477498
paper_authors.avatarUrl: /avatars/6bd01b4a6ea1e3a51d739d4104eb4ca9.svg
paper_authors.fullname: Gal Yona
paper_authors.type: user
paper_authors.user: galyona
paper_authors._id: 64618f4f2815a070474da2af
paper_authors.name: Or Honovich
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-15T18:49:59.258Z
paper_authors._id: 6173f5770a97dd394cbb2d99
paper_authors.avatarUrl: /avatars/ae71e4dc168ff3929a9f1ba02369e627.svg
paper_authors.fullname: Or Honovich
paper_authors.type: user
paper_authors.user: orhonovich
paper_authors._id: 64618f4f2815a070474da2b0
paper_authors.name: Itay Laish
paper_authors._id: 64618f4f2815a070474da2b1
paper_authors.name: Roee Aharoni
paper_title: Surfacing Biases in Large Language Models using Contrastive Input
    Decoding
paper_summary: Ensuring that large language models (LMs) are fair, robust and useful
  requires an understanding of how different modifications to their inputs impact
  the model's behaviour. In the context of open-text generation tasks, however,
  such an evaluation is not trivial. For example, when introducing a model with
  an input text and a perturbed, "contrastive" version of it, meaningful
  differences in the next-token predictions may not be revealed with standard
  decoding strategies. With this motivation in mind, we propose Contrastive Input
  Decoding (CID): a decoding algorithm to generate text given two inputs, where
  the generated text is likely given one input but unlikely given the other. In
  this way, the contrastive generations can highlight potentially subtle
  differences in how the LM output differs for the two inputs in a simple and
  interpretable manner. We use CID to highlight context-specific biases that are
  hard to detect with standard decoding strategies and quantify the effect of
  different input perturbations.
paper_ai_summary: Contrastive Input Decoding is proposed to generate text highlighting subtle differences in large language model outputs given contrasting inputs.
paper_ai_keywords: Contrastive Input Decoding CID large language models input text perturbed inputs next-token predictions context-specific biases