title: Current Pathology Foundation Models are unrobust to Medical Center
    Differences
summary: Pathology Foundation Models (FMs) hold great promise for healthcare. Before
  they can be used in clinical practice, it is essential to ensure they are
  robust to variations between medical centers. We measure whether pathology FMs
  focus on biological features like tissue and cancer type, or on the well known
  confounding medical center signatures introduced by staining procedure and
  other differences. We introduce the Robustness Index. This novel robustness
  metric reflects to what degree biological features dominate confounding
  features. Ten current publicly available pathology FMs are evaluated. We find
  that all current pathology foundation models evaluated represent the medical
  center to a strong degree. Significant differences in the robustness index are
  observed. Only one model so far has a robustness index greater than one,
  meaning biological features dominate confounding features, but only slightly. A
  quantitative approach to measure the influence of medical center differences on
  FM-based prediction performance is described. We analyze the impact of
  unrobustness on classification performance of downstream models, and find that
  cancer-type classification errors are not random, but specifically attributable
  to same-center confounders: images of other classes from the same medical
  center. We visualize FM embedding spaces, and find these are more strongly
  organized by medical centers than by biological factors. As a consequence, the
  medical center of origin is predicted more accurately than the tissue source
  and cancer type. The robustness index introduced here is provided with the aim
  of advancing progress towards clinical adoption of robust and reliable
  pathology FMs.
paper_authors._id: 67a197099b2f48315e74dcde
paper_authors.name: Edwin D. de Jong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T09:39:28.120Z
paper_authors._id: 67225dd94201755d88e104c4
paper_authors.avatarUrl: /avatars/6da69788ce0cd41c86f9dd0bf8d092aa.svg
paper_authors.fullname: Edwin D. de Jong
paper_authors.type: user
paper_authors.user: EdwinDdeJong
paper_authors._id: 67a197099b2f48315e74dcdf
paper_authors.name: Eric Marcus
paper_authors._id: 67a197099b2f48315e74dce0
paper_authors.name: Jonas Teuwen
paper_title: Current Pathology Foundation Models are unrobust to Medical Center
    Differences
paper_summary: Pathology Foundation Models (FMs) hold great promise for healthcare. Before
  they can be used in clinical practice, it is essential to ensure they are
  robust to variations between medical centers. We measure whether pathology FMs
  focus on biological features like tissue and cancer type, or on the well known
  confounding medical center signatures introduced by staining procedure and
  other differences. We introduce the Robustness Index. This novel robustness
  metric reflects to what degree biological features dominate confounding
  features. Ten current publicly available pathology FMs are evaluated. We find
  that all current pathology foundation models evaluated represent the medical
  center to a strong degree. Significant differences in the robustness index are
  observed. Only one model so far has a robustness index greater than one,
  meaning biological features dominate confounding features, but only slightly. A
  quantitative approach to measure the influence of medical center differences on
  FM-based prediction performance is described. We analyze the impact of
  unrobustness on classification performance of downstream models, and find that
  cancer-type classification errors are not random, but specifically attributable
  to same-center confounders: images of other classes from the same medical
  center. We visualize FM embedding spaces, and find these are more strongly
  organized by medical centers than by biological factors. As a consequence, the
  medical center of origin is predicted more accurately than the tissue source
  and cancer type. The robustness index introduced here is provided with the aim
  of advancing progress towards clinical adoption of robust and reliable
  pathology FMs.
paper_ai_summary: A novel robustness metric, the Robustness Index, evaluates the influence of medical center confounders on pathology foundation models, highlighting the need for improved biological feature focus.
paper_ai_keywords: Pathology Foundation Models Robustness Index medical center signatures biological features confounding features FM embedding spaces cancer-type classification tissue source