title: CXReasonBench: A Benchmark for Evaluating Structured Diagnostic
    Reasoning in Chest X-rays
summary: Recent progress in Large Vision-Language Models (LVLMs) has enabled promising
  applications in medical tasks, such as report generation and visual question
  answering. However, existing benchmarks focus mainly on the final diagnostic
  answer, offering limited insight into whether models engage in clinically
  meaningful reasoning. To address this, we present CheXStruct and CXReasonBench,
  a structured pipeline and benchmark built on the publicly available
  MIMIC-CXR-JPG dataset. CheXStruct automatically derives a sequence of
  intermediate reasoning steps directly from chest X-rays, such as segmenting
  anatomical regions, deriving anatomical landmarks and diagnostic measurements,
  computing diagnostic indices, and applying clinical thresholds. CXReasonBench
  leverages this pipeline to evaluate whether models can perform clinically valid
  reasoning steps and to what extent they can learn from structured guidance,
  enabling fine-grained and transparent assessment of diagnostic reasoning. The
  benchmark comprises 18,988 QA pairs across 12 diagnostic tasks and 1,200 cases,
  each paired with up to 4 visual inputs, and supports multi-path, multi-stage
  evaluation including visual grounding via anatomical region selection and
  diagnostic measurements. Even the strongest of 10 evaluated LVLMs struggle with
  structured reasoning and generalization, often failing to link abstract
  knowledge with anatomically grounded visual interpretation. The code is
  available at https://github.com/ttumyche/CXReasonBench
paper_authors._id: 68351b91037632c04211688d
paper_authors.name: Hyungyung Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:44:18.221Z
paper_authors._id: 6837b9a63ed37b18326c7fff
paper_authors.avatarUrl: /avatars/7aaf5e1ff783aad45946c937ac36bdd8.svg
paper_authors.fullname: Hyungyung Lee
paper_authors.type: user
paper_authors.user: ttumyche
paper_authors._id: 68351b91037632c04211688e
paper_authors.name: Geon Choi
paper_authors._id: 68351b91037632c04211688f
paper_authors.name: Jung-Oh Lee
paper_authors._id: 68351b91037632c042116890
paper_authors.name: Hangyul Yoon
paper_authors._id: 68351b91037632c042116891
paper_authors.name: Hyuk Gi Hong
paper_authors._id: 68351b91037632c042116892
paper_authors.name: Edward Choi
paper_title: CXReasonBench: A Benchmark for Evaluating Structured Diagnostic
    Reasoning in Chest X-rays
paper_summary: Recent progress in Large Vision-Language Models (LVLMs) has enabled promising
  applications in medical tasks, such as report generation and visual question
  answering. However, existing benchmarks focus mainly on the final diagnostic
  answer, offering limited insight into whether models engage in clinically
  meaningful reasoning. To address this, we present CheXStruct and CXReasonBench,
  a structured pipeline and benchmark built on the publicly available
  MIMIC-CXR-JPG dataset. CheXStruct automatically derives a sequence of
  intermediate reasoning steps directly from chest X-rays, such as segmenting
  anatomical regions, deriving anatomical landmarks and diagnostic measurements,
  computing diagnostic indices, and applying clinical thresholds. CXReasonBench
  leverages this pipeline to evaluate whether models can perform clinically valid
  reasoning steps and to what extent they can learn from structured guidance,
  enabling fine-grained and transparent assessment of diagnostic reasoning. The
  benchmark comprises 18,988 QA pairs across 12 diagnostic tasks and 1,200 cases,
  each paired with up to 4 visual inputs, and supports multi-path, multi-stage
  evaluation including visual grounding via anatomical region selection and
  diagnostic measurements. Even the strongest of 10 evaluated LVLMs struggle with
  structured reasoning and generalization, often failing to link abstract
  knowledge with anatomically grounded visual interpretation. The code is
  available at https://github.com/ttumyche/CXReasonBench
paper_ai_summary: CheXStruct and CXReasonBench evaluate Large Vision-Language Models in clinical diagnosis by assessing structured reasoning, visual grounding, and generalization using the MIMIC-CXR-JPG dataset.
paper_ai_keywords: Large Vision-Language Models LVLMs CheXStruct CXReasonBench MIMIC-CXR-JPG structured reasoning visual question answering medical tasks report generation anatomical regions diagnostic measurements diagnostic indices clinical thresholds visual grounding diagnostic reasoning multi-path multi-stage evaluation