How big is dataset for testing?
30 terms, each term-sanse pair has around 15 samples for testing
Automated analysis of clinical notes is attracting increasing attention. However, there has not been much work on medical term abbreviation disambiguation. Such abbreviations are abundant, and highly ambiguous, in clinical documents. One of the main obstacles is the lack of large scale, balance labeled data sets. To address the issue, we propose a few-shot learning approach to take advantage of limited labeled data. Specifically, a neural topic-attention model is applied to learn improved contextualized sentence representations for medical term abbreviation disambiguation. Another vital issue is that the existing scarce annotations are noisy and missing. We re-examine and correct an existing dataset for training and collect a test set to evaluate the models fairly especially for rare senses. We train our model on the training set which contains 30 abbreviation terms as categories (on average, 479 samples and 3.24 classes in each term) selected from a public abbreviation disambiguation dataset, and then test on a manually-created balanced dataset (each class in each term has 15 samples). We show that enhancing the sentence representation with topic information improves the performance on small-scale unbalanced training datasets by a large margin, compared to a number of baseline models.
Medical text mining is an exciting area and is becoming attractive to natural language processing (NLP) researchers. Clinical notes are an example of text in the medical area that recent work has focused on BIBREF0, BIBREF1, BIBREF2. This work studies abbreviation disambiguation on clinical notes BIBREF3, BIBREF4, specifically those used commonly by physicians and nurses. Such clinical abbreviations can have a large number of meanings, depending on the specialty BIBREF5, BIBREF6. For example, the term MR can mean magnetic resonance, mitral regurgitation, mental retardation, medical record and the general English Mister (Mr.). Table TABREF1 illustrates such an example. Abbreviation disambiguation is an important task in medical text understanding task BIBREF7. Successful recognition of the abbreviations in the notes can contribute to downstream tasks such as classification, named entity recognition, and relation extraction BIBREF7. Recent work focuses on formulating the abbreviation disambiguation task as a classification problem, where the possible senses of a given abbreviation term are pre-defined with the help of domain experts BIBREF6, BIBREF5. Traditional features such as part-of-speech (POS) and Term Frequency-Inverse Document Frequency (TF-IDF) are widely investigated for clinical notes classification. Classifiers like support vector machines (SVMs) and random forests (RFs) are used to make predictions BIBREF1. Such methods depend heavily on feature engineering. Recently, deep features have been investigated in the medical domain. Word embeddings BIBREF8 and Convolutional Neural Networks (CNNs) BIBREF9, BIBREF10 provide very competitive performance on text classification for clinical notes and abbreviation disambiguation task BIBREF0, BIBREF11, BIBREF12, BIBREF6. Beyond vanilla embeddings, BIBREF13 utilized contextual features to do abbreviation expansion. Another challenge is the difficulty in obtaining training data: clinical notes have many restrictions due to privacy issues and require domain experts to develop high-quality annotations, thus leading to limited annotated training and testing data. Another difficulty is that in the real world (and in the existing public datasets), some abbreviation term-sense pairs (for example, AB as abortion) have a very high frequency of occurrence BIBREF5, while others are rarely found. This long tail issue creates the challenge of training under unbalanced sample distributions. We tackle these problems in the setting of few-shot learning BIBREF14, BIBREF15 where only a few or a low number of samples can be found in the training dataset to make use of limited resources. We propose a model that combines deep contextual features based on ELMo BIBREF16 and topic information to solve the abbreviation disambiguation task. Our contributions can be summarized as: 1) we re-examined and corrected an existing dataset for training and we collected a test set for evaluation with focus especially for rare senses; 2) we proposed a few-shot learning approach which combines topic information and contextualized word embeddings to solve clinical abbreviation disambiguation task. The implementation is available online; 3) as limited research are conducted on this particular task, we evaluated and compared a number of baseline methods including classical models and deep models comprehensively.