title: Attention IoU: Examining Biases in CelebA using Attention Maps
summary: Computer vision models have been shown to exhibit and amplify biases across a
  wide array of datasets and tasks. Existing methods for quantifying bias in
  classification models primarily focus on dataset distribution and model
  performance on subgroups, overlooking the internal workings of a model. We
  introduce the Attention-IoU (Attention Intersection over Union) metric and
  related scores, which use attention maps to reveal biases within a model's
  internal representations and identify image features potentially causing the
  biases. First, we validate Attention-IoU on the synthetic Waterbirds dataset,
  showing that the metric accurately measures model bias. We then analyze the
  CelebA dataset, finding that Attention-IoU uncovers correlations beyond
  accuracy disparities. Through an investigation of individual attributes through
  the protected attribute of Male, we examine the distinct ways biases are
  represented in CelebA. Lastly, by subsampling the training set to change
  attribute correlations, we demonstrate that Attention-IoU reveals potential
  confounding variables not present in dataset labels.
paper_authors._id: 67e4c449672b3d9c9cb8aa4b
paper_authors.name: Aaron Serianni
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-27T09:03:27.573Z
paper_authors._id: 67e4c444692ba54248a6b337
paper_authors.avatarUrl: /avatars/7695a30d185ccb2354129c72538e9180.svg
paper_authors.fullname: Aaron Serianni
paper_authors.type: user
paper_authors.user: serianni
paper_authors._id: 67e4c449672b3d9c9cb8aa4c
paper_authors.name: Tyler Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:16:06.286Z
paper_authors._id: 6301af7259ab5d9dc0a15060
paper_authors.avatarUrl: /avatars/0bcda98801877332893a1de169000737.svg
paper_authors.fullname: Tyler Zhu
paper_authors.type: user
paper_authors.user: tyleryzhu
paper_authors._id: 67e4c449672b3d9c9cb8aa4d
paper_authors.name: Olga Russakovsky
paper_authors._id: 67e4c449672b3d9c9cb8aa4e
paper_authors.name: Vikram V. Ramaswamy
paper_title: Attention IoU: Examining Biases in CelebA using Attention Maps
paper_summary: Computer vision models have been shown to exhibit and amplify biases across a
  wide array of datasets and tasks. Existing methods for quantifying bias in
  classification models primarily focus on dataset distribution and model
  performance on subgroups, overlooking the internal workings of a model. We
  introduce the Attention-IoU (Attention Intersection over Union) metric and
  related scores, which use attention maps to reveal biases within a model's
  internal representations and identify image features potentially causing the
  biases. First, we validate Attention-IoU on the synthetic Waterbirds dataset,
  showing that the metric accurately measures model bias. We then analyze the
  CelebA dataset, finding that Attention-IoU uncovers correlations beyond
  accuracy disparities. Through an investigation of individual attributes through
  the protected attribute of Male, we examine the distinct ways biases are
  represented in CelebA. Lastly, by subsampling the training set to change
  attribute correlations, we demonstrate that Attention-IoU reveals potential
  confounding variables not present in dataset labels.
paper_ai_summary: The Attention-IoU metric uses attention maps to uncover biases within model's internal representations, identifying image features causing bias and revealing potential confounding variables.
paper_ai_keywords: Attention-IoU Attention Intersection over Union attention maps model bias internal representations image features Waterbirds dataset CelebA dataset attribute correlations protected attribute Male confounding variables