Diana MacLean|Jeffrey Heer
Identifying medical terms in patient-authored text: a crowdsourcing-based approach
Stanford University|Stanford University
background objective people increasingly engage online healthseeking behavior contribute healthoriented websites volume medical text authored patients medical novices grows rapidly however lack effective method automatically identifying medical terms patientauthored text pat demonstrate crowdsourcing pat medical term identification tasks nonexperts viable method creating large accuratelylabeled pat datasets moreover datasets used train classifiers outperform existing medical term identification tools materials methods evaluate viability using nonexpert crowds label pat compare expert registered nurses nonexpert amazon mechanical turk workers turkers responses pat medical term identification task next build crowdlabeled dataset comprising 10 000 sentences medhelp train two models dataset evaluate performance well metamap open biomedical annotator oba nactems termine two gold standard datasets one medhelp curetogether results aggregated according corroborative voting policy turker responses predict expert responses f1 score 84 conditional random field crf trained 10 000 crowdlabeled medhelp sentences achieves f1 score 78 curetogether gold standard widely outperforming oba 47 termine 43 metamap 39 failure analysis crf suggests misclassified terms likely either generic rare conclusions results show combining statistical models sensitive sentencelevel context crowdlabeled data scalable effective technique automatically identifying medical terms pat