title: RadEdit: stress-testing biomedical vision models via diffusion image
    editing
summary: Biomedical imaging datasets are often small and biased, meaning that
  real-world performance of predictive models can be substantially lower than
  expected from internal testing. This work proposes using generative image
  editing to simulate dataset shifts and diagnose failure modes of biomedical
  vision models; this can be used in advance of deployment to assess readiness,
  potentially reducing cost and patient harm. Existing editing methods can
  produce undesirable changes, with spurious correlations learned due to the
  co-occurrence of disease and treatment interventions, limiting practical
  applicability. To address this, we train a text-to-image diffusion model on
  multiple chest X-ray datasets and introduce a new editing method RadEdit that
  uses multiple masks, if present, to constrain changes and ensure consistency in
  the edited images. We consider three types of dataset shifts: acquisition
  shift, manifestation shift, and population shift, and demonstrate that our
  approach can diagnose failures and quantify model robustness without additional
  data collection, complementing more qualitative tools for explainable AI.
paper_authors._id: 6583c5892ae92808739a98b4
paper_authors.name: Fernando Pérez-García
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:32:51.738Z
paper_authors._id: 6278fbec17c069c72a11f35d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669369101851-6278fbec17c069c72a11f35d.jpeg
paper_authors.fullname: Fernando Pérez-García
paper_authors.type: user
paper_authors.user: fepegar
paper_authors._id: 6583c5892ae92808739a98b5
paper_authors.name: Sam Bond-Taylor
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-16T14:16:32.757Z
paper_authors._id: 65a65f755c58475cf9b2ba2f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65a65f755c58475cf9b2ba2f/kmjbPo_dxuklQSXKu1Jz5.jpeg
paper_authors.fullname: Sam Bond-Taylor
paper_authors.type: user
paper_authors.user: sambt
paper_authors._id: 6583c5892ae92808739a98b6
paper_authors.name: Pedro P. Sanchez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:33:57.158Z
paper_authors._id: 633bf51720ceb9fcae5abda3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633bf51720ceb9fcae5abda3/2GlqQb-EPv_MiR3xraBen.jpeg
paper_authors.fullname: Pedro Sanchez
paper_authors.type: user
paper_authors.user: PedroSanchez
paper_authors._id: 6583c5892ae92808739a98b7
paper_authors.name: Boris van Breugel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:34:05.112Z
paper_authors._id: 631e4567b9a0b2ff1cbe3e41
paper_authors.avatarUrl: /avatars/f0cd0394407b08d837ad2694e69c965a.svg
paper_authors.fullname: Boris van Breugel
paper_authors.type: user
paper_authors.user: vanbreugel
paper_authors._id: 6583c5892ae92808739a98b8
paper_authors.name: Daniel C. Castro
paper_authors._id: 6583c5892ae92808739a98b9
paper_authors.name: Harshita Sharma
paper_authors._id: 6583c5892ae92808739a98ba
paper_authors.name: Valentina Salvatelli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:35:12.058Z
paper_authors._id: 640dfb0bc364a086c633456e
paper_authors.avatarUrl: /avatars/bd28f9e14263a900a9b2f9f1f3479f1f.svg
paper_authors.fullname: Valentina Salvatelli
paper_authors.type: user
paper_authors.user: vsalvatelli
paper_authors._id: 6583c5892ae92808739a98bb
paper_authors.name: Maria T. A. Wetscherek
paper_authors._id: 6583c5892ae92808739a98bc
paper_authors.name: Hannah Richardson
paper_authors._id: 6583c5892ae92808739a98bd
paper_authors.name: Matthew P. Lungren
paper_authors._id: 6583c5892ae92808739a98be
paper_authors.name: Aditya Nori
paper_authors._id: 6583c5892ae92808739a98bf
paper_authors.name: Javier Alvarez-Valle
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:35:49.454Z
paper_authors._id: 6278d875bba7c62cbaaf7693
paper_authors.avatarUrl: /avatars/8d93e7c33acdfac853317d8fb5b0d485.svg
paper_authors.fullname: Javier Alvarez Valle
paper_authors.type: user
paper_authors.user: jaalvare
paper_authors._id: 6583c5892ae92808739a98c0
paper_authors.name: Ozan Oktay
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:35:56.318Z
paper_authors._id: 6274dd3eb43ccfeef481ce29
paper_authors.avatarUrl: /avatars/9f5d2a25a74af3cb0a227f946041206d.svg
paper_authors.fullname: Ozan Oktay
paper_authors.type: user
paper_authors.user: ozanoktay
paper_authors._id: 6583c5892ae92808739a98c1
paper_authors.name: Maximilian Ilse
paper_title: RadEdit: stress-testing biomedical vision models via diffusion image
    editing
paper_summary: Biomedical imaging datasets are often small and biased, meaning that
  real-world performance of predictive models can be substantially lower than
  expected from internal testing. This work proposes using generative image
  editing to simulate dataset shifts and diagnose failure modes of biomedical
  vision models; this can be used in advance of deployment to assess readiness,
  potentially reducing cost and patient harm. Existing editing methods can
  produce undesirable changes, with spurious correlations learned due to the
  co-occurrence of disease and treatment interventions, limiting practical
  applicability. To address this, we train a text-to-image diffusion model on
  multiple chest X-ray datasets and introduce a new editing method RadEdit that
  uses multiple masks, if present, to constrain changes and ensure consistency in
  the edited images. We consider three types of dataset shifts: acquisition
  shift, manifestation shift, and population shift, and demonstrate that our
  approach can diagnose failures and quantify model robustness without additional
  data collection, complementing more qualitative tools for explainable AI.
paper_ai_keywords: text-to-image diffusion model RadEdit masks acquisition shift manifestation shift population shift model robustness explainable AI