MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical Domain
<|reference_start|>MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical Domain: Medical texts are notoriously challenging to read. Properly measuring their readability is the first step towards making them more accessible. In this paper, we present a systematic study on fine-grained readability measurements in the medical domain at both sentence-level and span-level. We introduce a new dataset MedReadMe, which consists of manually annotated readability ratings and fine-grained complex span annotation for 4,520 sentences, featuring two novel "Google-Easy" and "Google-Hard" categories. It supports our quantitative analysis, which covers 650 linguistic features and automatic complex word and jargon identification. Enabled by our high-quality annotation, we benchmark and improve several state-of-the-art sentence-level readability metrics for the medical domain specifically, which include unsupervised, supervised, and prompting-based methods using recently developed large language models (LLMs). Informed by our fine-grained complex span annotation, we find that adding a single feature, capturing the number of jargon spans, into existing readability formulas can significantly improve their correlation with human judgments. We will publicly release the dataset and code.<|reference_end|>
@article{jiang2024medreadme:,
    title={MedReadMe: A Systematic Study for Fine-grained Sentence Readability in
  Medical Domain},
    author={Chao Jiang and Wei Xu},
    journal={arXiv preprint arXiv:2405.02144},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.02144},
    primaryClass={cs.CL}
}