TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers
<|reference_start|>TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers: Text segmentation holds paramount importance in the field of Natural Language Processing (NLP). It plays an important role in several NLP downstream tasks like information retrieval and document summarization. In this work, we propose a new solution, namely TocBERT, for segmenting texts using bidirectional transformers. TocBERT represents a supervised solution trained on the detection of titles and sub-titles from their semantic representations. This task was formulated as a named entity recognition (NER) problem. The solution has been applied on a medical text segmentation use-case where the Bio-ClinicalBERT model is fine-tuned to segment discharge summaries of the MIMIC-III dataset. The performance of TocBERT has been evaluated on a human-labeled ground truth corpus of 250 notes. It achieved an F1-score of 84.6% when evaluated on a linear text segmentation problem and 72.8% on a hierarchical text segmentation problem. It outperformed a carefully designed rule-based solution, particularly in distinguishing titles from subtitles.<|reference_end|>
@article{saleh2024tocbert:,
    title={TocBERT: Medical Document Structure Extraction Using Bidirectional
  Transformers},
    author={Majd Saleh and Sarra Baghdadi and St'ephane Paquelet},
    journal={arXiv preprint arXiv:2406.19526},
    year={2024},
    number={The article has been accepted for publication in the 12th IEEE
  International Conference on Intelligent Systems 2024},
    archivePrefix={arXiv},
    eprint={2406.19526},
    primaryClass={cs.CL cs.IR cs.LG}
}