Sarah Blackstone, Fei Luo, R Michael Barker, Rose A Sevcik, MaryAnn Romski, Vicki Casella, Christine Roman-Lantzy
Profiles of Children With Cortical Visual Impairment Who Use Augmentative and Alternative Communication: A Retrospective Examination
American journal of speech-language pathology
error analysis plays crucial role clinical concept extraction fundamental subtask within clinical natural language processing nlp process typically involves manual review error types contextual linguistic factors contributing occurrence identification underlying causes refine nlp model improve performance conducting error analysis complex requiring combination nlp expertise domainspecific knowledge due high heterogeneity electronic health record ehr settings across different institutions challenges may arise attempting standardize reproduce error analysis process study aims facilitate collaborative effort establish common definitions taxonomies capturing diverse error types fostering community consensus error analysis clinical concept extraction tasks iteratively developed evaluated error taxonomy based existing literature standards realworld data multisite case evaluations community feedback finalized taxonomy released dtd owl formats open health natural language processing consortium taxonomy compatible several different opensource annotation tools including mae brat medtator resulting error taxonomy comprises 43 distinct error classes organized 6 error dimensions 4 properties including model type symbolic statistical machine learning evaluation subject model human evaluation level patient document sentence concept annotation examples internal external evaluations revealed strong variations error types across methodological approaches tasks ehr settings key points emerged community feedback including need enhancing clarity generalizability usability taxonomy along dissemination strategies proposed taxonomy facilitate acceleration standardization error analysis process multisite settings thus improving provenance interpretability portability nlp models future researchers could explore potential direction developing automated semiautomated methods assist classification standardization error analysis