Stig-Johan Wiklund, Katharine Thorn, Heiko Götte, Kimberley Hacquoil, Gaëlle Saint-Hilary, Alex Carlton
Going beyond probability of success: Opportunities for statisticians to influence quantitative decision-making at the portfolio level
Pharmaceutical statistics
social mediabased public health research crucial epidemic surveillance studies identify relevant corpora keywordmatching study develops system streamline process curating colloquial medical dictionaries demonstrate pipeline curating umlscolloquial symptom dictionary covid19related tweets proof concept covid19related tweets february 1 2020 april 30 2022 used pipeline includes three modules named entity recognition module detect symptoms tweets entity normalization module aggregate detected entities mapping module iteratively maps entities unified medical language system concepts random 500 entity sample drawn final dictionary accuracy validation additionally conducted symptom frequency distribution analysis compare dictionary predefined lexicon previous research identified 498480 unique symptom entity expressions tweets preprocessing reduces number 18226 final dictionary contains 38175 unique expressions symptoms mapped 966 umls concepts accuracy 95 symptom distribution analysis found dictionary detects symptoms effective identifying psychiatric disorders like anxiety depression often missed predefined lexicons study advances public health research implementing novel systematic pipeline curating symptom lexicons social media data final lexicons high accuracy validated medical professionals underscores potential methodology reliably interpret categorize vast amounts unstructured social media data actionable medical insights across diverse linguistic regional landscapes