title: Named Clinical Entity Recognition Benchmark
summary: This technical report introduces a Named Clinical Entity Recognition
  Benchmark for evaluating language models in healthcare, addressing the crucial
  natural language processing (NLP) task of extracting structured information
  from clinical narratives to support applications like automated coding,
  clinical trial cohort identification, and clinical decision support.
    The leaderboard provides a standardized platform for assessing diverse
  language models, including encoder and decoder architectures, on their ability
  to identify and classify clinical entities across multiple medical domains. A
  curated collection of openly available clinical datasets is utilized,
  encompassing entities such as diseases, symptoms, medications, procedures, and
  laboratory measurements. Importantly, these entities are standardized according
  to the Observational Medical Outcomes Partnership (OMOP) Common Data Model,
  ensuring consistency and interoperability across different healthcare systems
  and datasets, and a comprehensive evaluation of model performance. Performance
  of models is primarily assessed using the F1-score, and it is complemented by
  various assessment modes to provide comprehensive insights into model
  performance. The report also includes a brief analysis of models evaluated to
  date, highlighting observed trends and limitations.
    By establishing this benchmarking framework, the leaderboard aims to promote
  transparency, facilitate comparative analyses, and drive innovation in clinical
  entity recognition tasks, addressing the need for robust evaluation methods in
  healthcare NLP.
paper_authors._id: 67050c6448db87e174db722b
paper_authors.name: Wadood M Abdul
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-09T07:37:56.785Z
paper_authors._id: 5f27a163e923d665e616271c
paper_authors.avatarUrl: /avatars/2b10d5846cef8a01b5db4c435b6f2628.svg
paper_authors.fullname: Wadood Abdul
paper_authors.type: user
paper_authors.user: wadood
paper_authors._id: 67050c6448db87e174db722c
paper_authors.name: Marco AF Pimentel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-08T12:45:56.120Z
paper_authors._id: 6454faafa13edf669cd74f36
paper_authors.avatarUrl: /avatars/355d16e28ca9cf5891368e43bcda6de5.svg
paper_authors.fullname: Marco Pimentel
paper_authors.type: user
paper_authors.user: mpimentel
paper_authors._id: 67050c6448db87e174db722d
paper_authors.name: Muhammad Umar Salman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-09T13:16:59.116Z
paper_authors._id: 65f012b513700cbfce47c0f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65f012b513700cbfce47c0f9/8EW1TkevArZOaWHYOZ859.jpeg
paper_authors.fullname: Muhammad Umar Salman
paper_authors.type: user
paper_authors.user: umarsalman
paper_authors._id: 67050c6448db87e174db722e
paper_authors.name: Tathagata Raha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-09T13:17:05.096Z
paper_authors._id: 5f5f6c113c67af20d9945afb
paper_authors.avatarUrl: /avatars/06b2eb3a5d27864280d4d02e6d00d782.svg
paper_authors.fullname: Tathagata Raha
paper_authors.type: user
paper_authors.user: tathagataraha
paper_authors._id: 67050c6448db87e174db722f
paper_authors.name: Clément Christophe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-09T13:17:11.030Z
paper_authors._id: 628e39f4b1596566033b8d7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/628e39f4b1596566033b8d7b/-Y807up1cgMmAQsczdOPn.jpeg
paper_authors.fullname: Clément Christophe
paper_authors.type: user
paper_authors.user: cchristophe
paper_authors._id: 67050c6448db87e174db7230
paper_authors.name: Praveen K Kanithi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-08T12:46:16.946Z
paper_authors._id: 65280984b794fe3d06544d77
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65280984b794fe3d06544d77/tyrxbxtDG02On1uiRaVbL.jpeg
paper_authors.fullname: Praveenkumar
paper_authors.type: user
paper_authors.user: pkanithi
paper_authors._id: 67050c6448db87e174db7231
paper_authors.name: Nasir Hayat
paper_authors._id: 67050c6448db87e174db7232
paper_authors.name: Ronnie Rajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-09T13:17:31.565Z
paper_authors._id: 65281d6ef61ca80b9c2ee707
paper_authors.avatarUrl: /avatars/090ea7210a4bb6549b0f7fee71525625.svg
paper_authors.fullname: Ronnie Rajan
paper_authors.type: user
paper_authors.user: ronnierajan
paper_authors._id: 67050c6448db87e174db7233
paper_authors.name: Shadab Khan
paper_title: Named Clinical Entity Recognition Benchmark
paper_summary: This technical report introduces a Named Clinical Entity Recognition
  Benchmark for evaluating language models in healthcare, addressing the crucial
  natural language processing (NLP) task of extracting structured information
  from clinical narratives to support applications like automated coding,
  clinical trial cohort identification, and clinical decision support.
    The leaderboard provides a standardized platform for assessing diverse
  language models, including encoder and decoder architectures, on their ability
  to identify and classify clinical entities across multiple medical domains. A
  curated collection of openly available clinical datasets is utilized,
  encompassing entities such as diseases, symptoms, medications, procedures, and
  laboratory measurements. Importantly, these entities are standardized according
  to the Observational Medical Outcomes Partnership (OMOP) Common Data Model,
  ensuring consistency and interoperability across different healthcare systems
  and datasets, and a comprehensive evaluation of model performance. Performance
  of models is primarily assessed using the F1-score, and it is complemented by
  various assessment modes to provide comprehensive insights into model
  performance. The report also includes a brief analysis of models evaluated to
  date, highlighting observed trends and limitations.
    By establishing this benchmarking framework, the leaderboard aims to promote
  transparency, facilitate comparative analyses, and drive innovation in clinical
  entity recognition tasks, addressing the need for robust evaluation methods in
  healthcare NLP.