  Since radiology reports needed for clinical practice and research are written
and stored in free-text narrations, extraction of relative information for
further analysis is difficult. In these circumstances, natural language
processing (NLP) techniques can facilitate automatic information extraction and
transformation of free-text formats to structured data. In recent years, deep
learning (DL)-based models have been adapted for NLP experiments with promising
results. Despite the significant potential of DL models based on artificial
neural networks (ANN) and convolutional neural networks (CNN), the models face
some limitations to implement in clinical practice. Transformers, another new
DL architecture, have been increasingly applied to improve the process.
Therefore, in this study, we propose a transformer-based fine-grained named
entity recognition (NER) architecture for clinical information extraction. We
collected 88 abdominopelvic sonography reports in free-text formats and
annotated them based on our developed information schema. The text-to-text
transfer transformer model (T5) and Scifive, a pre-trained domain-specific
adaptation of the T5 model, were applied for fine-tuning to extract entities
and relations and transform the input into a structured format. Our
transformer-based model in this study outperformed previously applied
approaches such as ANN and CNN models based on ROUGE-1, ROUGE-2, ROUGE-L, and
BLEU scores of 0.816, 0.668, 0.528, and 0.743, respectively, while providing an
interpretable structured report.

Computation and Language;Artificial Intelligence;Machine Learning