title: What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine
    Translation with a Human-centered Study
summary: Gender bias in machine translation (MT) is recognized as an issue that can
  harm people and society. And yet, advancements in the field rarely involve
  people, the final MT users, or inform how they might be impacted by biased
  technologies. Current evaluations are often restricted to automatic methods,
  which offer an opaque estimate of what the downstream impact of gender
  disparities might be. We conduct an extensive human-centered study to examine
  if and to what extent bias in MT brings harms with tangible costs, such as
  quality of service gaps across women and men. To this aim, we collect
  behavioral data from 90 participants, who post-edited MT outputs to ensure
  correct gender translation. Across multiple datasets, languages, and types of
  users, our study shows that feminine post-editing demands significantly more
  technical and temporal effort, also corresponding to higher financial costs.
  Existing bias measurements, however, fail to reflect the found disparities. Our
  findings advocate for human-centered approaches that can inform the societal
  impact of bias.
paper_authors._id: 66fd206f9bd05d6ff6910ba3
paper_authors.name: Beatrice Savoldi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:36:43.306Z
paper_authors._id: 63b58c53bda8d44adf3092de
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63b58c53bda8d44adf3092de/7Qs7MmPHmQtZ7Hno5n6-P.jpeg
paper_authors.fullname: Beatrice Savoldi
paper_authors.type: user
paper_authors.user: BSavoldi
paper_authors._id: 66fd206f9bd05d6ff6910ba4
paper_authors.name: Sara Papi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:37:43.497Z
paper_authors._id: 66309b3833ccd9e68c5d5171
paper_authors.avatarUrl: /avatars/bd2a3fb38820c828bdb1acb93b673cb4.svg
paper_authors.fullname: Sara Papi
paper_authors.type: user
paper_authors.user: spapi
paper_authors._id: 66fd206f9bd05d6ff6910ba5
paper_authors.name: Matteo Negri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:37:04.221Z
paper_authors._id: 66350219843f549fdac86347
paper_authors.avatarUrl: /avatars/1ad7aa4e8e6d80c5d50bf4de502f11a4.svg
paper_authors.fullname: Matteo Negri
paper_authors.type: user
paper_authors.user: MNegri
paper_authors._id: 66fd206f9bd05d6ff6910ba6
paper_authors.name: Ana Guerberof
paper_authors._id: 66fd206f9bd05d6ff6910ba7
paper_authors.name: Luisa Bentivogli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:37:21.028Z
paper_authors._id: 662eb48f5ade870b60db4380
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ozHyP-GBIAEhR-l-TkBFb.jpeg
paper_authors.fullname: Luisa Bentivogli
paper_authors.type: user
paper_authors.user: lubentivogli
paper_title: What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine
    Translation with a Human-centered Study
paper_summary: Gender bias in machine translation (MT) is recognized as an issue that can
  harm people and society. And yet, advancements in the field rarely involve
  people, the final MT users, or inform how they might be impacted by biased
  technologies. Current evaluations are often restricted to automatic methods,
  which offer an opaque estimate of what the downstream impact of gender
  disparities might be. We conduct an extensive human-centered study to examine
  if and to what extent bias in MT brings harms with tangible costs, such as
  quality of service gaps across women and men. To this aim, we collect
  behavioral data from 90 participants, who post-edited MT outputs to ensure
  correct gender translation. Across multiple datasets, languages, and types of
  users, our study shows that feminine post-editing demands significantly more
  technical and temporal effort, also corresponding to higher financial costs.
  Existing bias measurements, however, fail to reflect the found disparities. Our
  findings advocate for human-centered approaches that can inform the societal
  impact of bias.
paper_ai_keywords: machine translation (MT) gender bias post-editing feminine post-editing technical effort temporal effort financial costs bias measurements societal impact