Named Clinical Entity Recognition Benchmark
This technical report introduces a Named Clinical Entity Recognition
Benchmark for evaluating language models in healthcare, addressing the crucial
natural language processing (NLP) task of extracting structured information
from clinical narratives to support applications like automated coding,
clinical trial cohort identification, and clinical decision support.
  The leaderboard provides a standardized platform for assessing diverse
language models, including encoder and decoder architectures, on their ability
to identify and classify clinical entities across multiple medical domains. A
curated collection of openly available clinical datasets is utilized,
encompassing entities such as diseases, symptoms, medications, procedures, and
laboratory measurements. Importantly, these entities are standardized according
to the Observational Medical Outcomes Partnership (OMOP) Common Data Model,
ensuring consistency and interoperability across different healthcare systems
and datasets, and a comprehensive evaluation of model performance. Performance
of models is primarily assessed using the F1-score, and it is complemented by
various assessment modes to provide comprehensive insights into model
performance. The report also includes a brief analysis of models evaluated to
date, highlighting observed trends and limitations.
  By establishing this benchmarking framework, the leaderboard aims to promote
transparency, facilitate comparative analyses, and drive innovation in clinical
entity recognition tasks, addressing the need for robust evaluation methods in
healthcare NLP.
67050c6448db87e174db722b
Wadood M Abdul
claimed_verified
2024-10-09T07:37:56.785Z
5f27a163e923d665e616271c
/avatars/2b10d5846cef8a01b5db4c435b6f2628.svg
Wadood Abdul
user
wadood
67050c6448db87e174db722c
Marco AF Pimentel
claimed_verified
2024-10-08T12:45:56.120Z
6454faafa13edf669cd74f36
/avatars/355d16e28ca9cf5891368e43bcda6de5.svg
Marco Pimentel
user
mpimentel
67050c6448db87e174db722d
Muhammad Umar Salman
admin_assigned
2024-10-09T13:16:59.116Z
65f012b513700cbfce47c0f9
https://cdn-avatars.huggingface.co/v1/production/uploads/65f012b513700cbfce47c0f9/8EW1TkevArZOaWHYOZ859.jpeg
Muhammad Umar Salman
user
umarsalman
67050c6448db87e174db722e
Tathagata Raha
admin_assigned
2024-10-09T13:17:05.096Z
5f5f6c113c67af20d9945afb
/avatars/06b2eb3a5d27864280d4d02e6d00d782.svg
Tathagata Raha
user
tathagataraha
67050c6448db87e174db722f
Clément Christophe
admin_assigned
2024-10-09T13:17:11.030Z
628e39f4b1596566033b8d7b
https://cdn-avatars.huggingface.co/v1/production/uploads/628e39f4b1596566033b8d7b/-Y807up1cgMmAQsczdOPn.jpeg
Clément Christophe
user
cchristophe
67050c6448db87e174db7230
Praveen K Kanithi
claimed_verified
2024-10-08T12:46:16.946Z
65280984b794fe3d06544d77
https://cdn-avatars.huggingface.co/v1/production/uploads/65280984b794fe3d06544d77/tyrxbxtDG02On1uiRaVbL.jpeg
Praveenkumar
user
pkanithi
67050c6448db87e174db7231
Nasir Hayat
67050c6448db87e174db7232
Ronnie Rajan
admin_assigned
2024-10-09T13:17:31.565Z
65281d6ef61ca80b9c2ee707
/avatars/090ea7210a4bb6549b0f7fee71525625.svg
Ronnie Rajan
user
ronnierajan
67050c6448db87e174db7233
Shadab Khan
Named Clinical Entity Recognition Benchmark
This technical report introduces a Named Clinical Entity Recognition
Benchmark for evaluating language models in healthcare, addressing the crucial
natural language processing (NLP) task of extracting structured information
from clinical narratives to support applications like automated coding,
clinical trial cohort identification, and clinical decision support.
  The leaderboard provides a standardized platform for assessing diverse
language models, including encoder and decoder architectures, on their ability
to identify and classify clinical entities across multiple medical domains. A
curated collection of openly available clinical datasets is utilized,
encompassing entities such as diseases, symptoms, medications, procedures, and
laboratory measurements. Importantly, these entities are standardized according
to the Observational Medical Outcomes Partnership (OMOP) Common Data Model,
ensuring consistency and interoperability across different healthcare systems
and datasets, and a comprehensive evaluation of model performance. Performance
of models is primarily assessed using the F1-score, and it is complemented by
various assessment modes to provide comprehensive insights into model
performance. The report also includes a brief analysis of models evaluated to
date, highlighting observed trends and limitations.
  By establishing this benchmarking framework, the leaderboard aims to promote
transparency, facilitate comparative analyses, and drive innovation in clinical
entity recognition tasks, addressing the need for robust evaluation methods in
healthcare NLP.
A benchmark for evaluating language models in clinical entity recognition uses the OMOP Common Data Model for standardized assessment of diverse architectures across multiple medical domains.
Named Clinical Entity Recognition encoder architectures decoder architectures clinical narratives automated coding clinical trial cohort identification clinical decision support F1-score