Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting
<|reference_start|>Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting: Large language models (LLMs) have become the preferred solution for many natural language processing tasks. In low-resource environments such as specialized domains, their few-shot capabilities are expected to deliver high performance. Named Entity Recognition (NER) is a critical task in information extraction that is not covered in recent LLM benchmarks. There is a need for better understanding the performance of LLMs for NER in a variety of settings including languages other than English. This study aims to evaluate generative LLMs, employed through prompt engineering, for few-shot clinical NER. %from the perspective of F1 performance and environmental impact. We compare 13 auto-regressive models using prompting and 16 masked models using fine-tuning on 14 NER datasets covering English, French and Spanish. While prompt-based auto-regressive models achieve competitive F1 for general NER, they are outperformed within the clinical domain by lighter biLSTM-CRF taggers based on masked models. Additionally, masked models exhibit lower environmental impact compared to auto-regressive models. Findings are consistent across the three languages studied, which suggests that LLM prompting is not yet suited for NER production in the clinical domain.<|reference_end|>
@article{naguib2024few-shot,
    title={Few-shot clinical entity recognition in English, French and Spanish:
  masked language models outperform generative model prompting},
    author={Marco Naguib, Xavier Tannier, Aur'elie N'ev'eol},
    journal={arXiv preprint arXiv:2402.12801},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12801},
    primaryClass={cs.CL}
}