title: BiasEdit: Debiasing Stereotyped Language Models via Model Editing
summary: Previous studies have established that language models manifest stereotyped
  biases. Existing debiasing strategies, such as retraining a model with
  counterfactual data, representation projection, and prompting often fail to
  efficiently eliminate bias or directly alter the models' biased internal
  representations. To address these issues, we propose BiasEdit, an efficient
  model editing method to remove stereotypical bias from language models through
  lightweight networks that act as editors to generate parameter updates.
  BiasEdit employs a debiasing loss guiding editor networks to conduct local
  edits on partial parameters of a language model for debiasing while preserving
  the language modeling abilities during editing through a retention loss.
  Experiments on StereoSet and Crows-Pairs demonstrate the effectiveness,
  efficiency, and robustness of BiasEdit in eliminating bias compared to
  tangental debiasing baselines and little to no impact on the language models'
  general capabilities. In addition, we conduct bias tracing to probe bias in
  various modules and explore bias editing impacts on different components of
  language models.
paper_authors._id: 67d125362da8f91f8ef0412f
paper_authors.name: Xin Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:36:07.130Z
paper_authors._id: 6190ab805ca89a28e9f66873
paper_authors.avatarUrl: /avatars/a677a8401360be473895494e5fb267bb.svg
paper_authors.fullname: Xin Xu
paper_authors.type: user
paper_authors.user: XinXuNLPer
paper_authors._id: 67d125362da8f91f8ef04130
paper_authors.name: Wei Xu
paper_authors._id: 67d125362da8f91f8ef04131
paper_authors.name: Ningyu Zhang
paper_authors._id: 67d125362da8f91f8ef04132
paper_authors.name: Julian McAuley
paper_title: BiasEdit: Debiasing Stereotyped Language Models via Model Editing
paper_summary: Previous studies have established that language models manifest stereotyped
  biases. Existing debiasing strategies, such as retraining a model with
  counterfactual data, representation projection, and prompting often fail to
  efficiently eliminate bias or directly alter the models' biased internal
  representations. To address these issues, we propose BiasEdit, an efficient
  model editing method to remove stereotypical bias from language models through
  lightweight networks that act as editors to generate parameter updates.
  BiasEdit employs a debiasing loss guiding editor networks to conduct local
  edits on partial parameters of a language model for debiasing while preserving
  the language modeling abilities during editing through a retention loss.
  Experiments on StereoSet and Crows-Pairs demonstrate the effectiveness,
  efficiency, and robustness of BiasEdit in eliminating bias compared to
  tangental debiasing baselines and little to no impact on the language models'
  general capabilities. In addition, we conduct bias tracing to probe bias in
  various modules and explore bias editing impacts on different components of
  language models.
paper_ai_keywords: language models counterfactual data representation projection prompting BiasEdit parameter updates debiasing loss retention loss StereoSet Crows-Pairs language modeling abilities bias tracing