Multi-objective Representation for Numbers in Clinical Narratives Using CamemBERT-bio
<|reference_start|>Multi-objective Representation for Numbers in Clinical Narratives Using CamemBERT-bio: This research aims to classify numerical values extracted from medical documents across seven distinct physiological categories, employing CamemBERT-bio. Previous studies suggested that transformer-based models might not perform as well as traditional NLP models in such tasks. To enhance CamemBERT-bio's performances, we introduce two main innovations: integrating keyword embeddings into the model and adopting a number-agnostic strategy by excluding all numerical data from the text. The implementation of label embedding techniques refines the attention mechanisms, while the technique of using a `numerical-blind' dataset aims to bolster context-centric learning. Another key component of our research is determining the criticality of extracted numerical data. To achieve this, we utilized a simple approach that involves verifying if the value falls within the established standard ranges. Our findings are encouraging, showing substantial improvements in the effectiveness of CamemBERT-bio, surpassing conventional methods with an F1 score of 0.89. This represents an over 20\% increase over the 0.73 $F_1$ score of traditional approaches and an over 9\% increase over the 0.82 $F_1$ score of state-of-the-art approaches. All this was achieved despite using small and imbalanced training datasets.<|reference_end|>
@article{lompo2024multi-objective,
    title={Multi-objective Representation for Numbers in Clinical Narratives Using
  CamemBERT-bio},
    author={Boammani Aser Lompo, Thanh-Dung Le},
    journal={arXiv preprint arXiv:2405.18448},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.18448},
    primaryClass={cs.CL eess.SP}
}