Lunguage: A Benchmark for Structured and Sequential Chest X-ray
  Interpretation
Radiology reports convey detailed clinical observations and capture
diagnostic reasoning that evolves over time. However, existing evaluation
methods are limited to single-report settings and rely on coarse metrics that
fail to capture fine-grained clinical semantics and temporal dependencies. We
introduce LUNGUAGE,a benchmark dataset for structured radiology report
generation that supports both single-report evaluation and longitudinal
patient-level assessment across multiple studies. It contains 1,473 annotated
chest X-ray reports, each reviewed by experts, and 80 of them contain
longitudinal annotations to capture disease progression and inter-study
intervals, also reviewed by experts. Using this benchmark, we develop a
two-stage framework that transforms generated reports into fine-grained,
schema-aligned structured representations, enabling longitudinal
interpretation. We also propose LUNGUAGESCORE, an interpretable metric that
compares structured outputs at the entity, relation, and attribute level while
modeling temporal consistency across patient timelines. These contributions
establish the first benchmark dataset, structuring framework, and evaluation
metric for sequential radiology reporting, with empirical results demonstrating
that LUNGUAGESCORE effectively supports structured report evaluation. The code
is available at: https://github.com/SuperSupermoon/Lunguage
6838be453538dad22f9bc2a4
Jong Hak Moon
claimed_verified
2025-05-30T06:58:33.487Z
63e50836c9b2b7e1ea3413fd
/avatars/6c43fbce3ac2564e7860ff33fe8c9e82.svg
Jonghak Moon
user
SuperSupermoon
6838be453538dad22f9bc2a5
Geon Choi
claimed_verified
2025-07-15T19:12:14.295Z
66714579562ee91c16be977c
/avatars/fd1a386c1e06fcbc9c06017d18e4fe5f.svg
Geon Choi
user
checkone
6838be453538dad22f9bc2a6
Paloma Rabaey
6838be453538dad22f9bc2a7
Min Gwan Kim
6838be453538dad22f9bc2a8
Hyuk Gi Hong
6838be453538dad22f9bc2a9
Jung-Oh Lee
6838be453538dad22f9bc2aa
Hangyul Yoon
6838be453538dad22f9bc2ab
Eun Woo Doe
6838be453538dad22f9bc2ac
Jiyoun Kim
6838be453538dad22f9bc2ad
Harshita Sharma
6838be453538dad22f9bc2ae
Daniel C. Castro
6838be453538dad22f9bc2af
Javier Alvarez-Valle
6838be453538dad22f9bc2b0
Edward Choi
Lunguage: A Benchmark for Structured and Sequential Chest X-ray
  Interpretation
Radiology reports convey detailed clinical observations and capture
diagnostic reasoning that evolves over time. However, existing evaluation
methods are limited to single-report settings and rely on coarse metrics that
fail to capture fine-grained clinical semantics and temporal dependencies. We
introduce LUNGUAGE,a benchmark dataset for structured radiology report
generation that supports both single-report evaluation and longitudinal
patient-level assessment across multiple studies. It contains 1,473 annotated
chest X-ray reports, each reviewed by experts, and 80 of them contain
longitudinal annotations to capture disease progression and inter-study
intervals, also reviewed by experts. Using this benchmark, we develop a
two-stage framework that transforms generated reports into fine-grained,
schema-aligned structured representations, enabling longitudinal
interpretation. We also propose LUNGUAGESCORE, an interpretable metric that
compares structured outputs at the entity, relation, and attribute level while
modeling temporal consistency across patient timelines. These contributions
establish the first benchmark dataset, structuring framework, and evaluation
metric for sequential radiology reporting, with empirical results demonstrating
that LUNGUAGESCORE effectively supports structured report evaluation. The code
is available at: https://github.com/SuperSupermoon/Lunguage
The paper introduces LUNGUAGE, a benchmark for structured radiology report generation, and LUNGUAGESCORE, an evaluation metric, enabling fine-grained structured report evaluation and longitudinal interpretation.
LUNGUAGE LUNGUAGESCORE structured radiology report generation two-stage framework schema-aligned structured representations evaluative metric entity relation attribute temporal consistency longitudinal interpretation