In traditional Distributional Semantic Models (DSMs) the multiple senses of a polysemous word are conflated into a single vector space representation. In this work, we propose a DSM that learns multiple distributional representations of a word based on different topics. First, a separate DSM is trained for each topic and then each of the topic-based DSMs is aligned to a common vector space. Our unsupervised mapping approach is motivated by the hypothesis that words preserving their relative distances in different topic semantic sub-spaces constitute robust semantic anchors that define the mappings between them. Aligned cross-topic representations achieve state-of-the-art results for the task of contextual word similarity. Furthermore, evaluation on NLP downstream tasks shows that multiple topic-based embeddings outperform single-prototype models. Word-level representation learning algorithms adopt the distributional hypothesis (Harris, 1954), presuming a correlation between the distributional and the semantic relationships of words. Typically, these models encode the contextual information of words into dense feature vectorsoften referred to as embeddings of a k -dimensional space, thus creating a Vector Space Model (VSM) of lexical semantics. Such embeddings have been successfully applied to various natural language processing applications, including information retrieval (Manning et al., 2008), sentiment analysis (Tai et al., 2015), and machine translation (Amiri et al., 2016; Sharaf et al., 2017). Despite their popularity, traditional DSMs rely solely on models where each word is uniquely represented by one point in the vector space. From a  The research was performed when the author was an undergraduate researcher at School of ECE, NTUA in Athens, Greece. linguistic perspective, these models cannot capture the distinct meanings of polysemous words (e.g., bank or cancer ), resulting in conflated word representations of diverse contextual semantics. To alleviate this problem, DSMs with multiple representations per word have been proposed in the literature, based on clustering local contexts of individual words (Reisinger and Mooney, 2010; Tian et al., 2014; Neelakantan et al., 2014). An alternative way to train multiple representation DSMs is to utilize semantic lexical resources (Rothe and Schutze, 2015; Pilehvar and Collier, 2016). Christopoulou et al. (2018), based on the assumption that typically words appear with a specific sense in each topic, proposed a topic-based semantic mixture model that exploits a combination of similarities estimated on topic-based DSMs for the computation of semantic similarity between words. Their model performs well for a variety of semantic similarity tasks; however, it lacks a unified representation of multiple senses in a common semantic space. The problem of defining transformations between embeddingstrained independently under different corporahas been previously examined in various works, such as machine translation (Mikolov et al., 2013b; Xing et al., 2015; Artetxe et al., 2016), induction of historical embeddings (Hamilton et al., 2016) and lexical resources enrichment (Prokhorov et al., 2017). Following this line of research, we induce the creation of multiple cross-topic word embeddings by projecting the semantic representations of topic-based DSMs to a unified semantic space. We investigate different ways to perform the mappings from the topic sub-spaces to the unified semantic space, and propose a completely unsupervised approach to extract semantic anchors that define those mappings. Furthermore, we claim that polysemous words change their meaning in different topic domains; this is reflected in relative shifts of their distributional representations in different topic-based DSMs. On the other hand, semantic anchors should have consistent semantic relationships regardless of the domain they reside in. Hence, their distributions of similarity values should also be similar across different domains. Finally, we apply a smoothing technique to each word's set of topic embeddings, resulting in representations with fine-grained semantics. To our knowledge, this is the first time that mappings between semantic spaces are applied to the problem of learning multiple embeddings for polysemous words. Our multi-topic word representations are evaluated on the contextual semantic similarity task and yield state-of-the-art performance compared to other unsupervised multi-prototype word embedding approaches. We further perform experiments on two NLP downstream tasks: text classification and paraphrase identification and demonstrate that our learned word representations consistently provide higher performance than single-prototype word embedding models. The code of the present work is publicly available 1 . Methods that assign multiple distributed representations per word can be grouped into two broad categories. 2 Unsupervised methods induce multiple word representations without leveraging semantic lexical resources. Reisinger and Mooney (2010) were the first to create a multi-prototype DSM with a fixed number of vectors assigned to each word. In their model, the centroids of context-dependent clusters were used to create a set of sense-specific vectors for each target word. Based on similar clustering approaches, follow-up works introduced neural network architectures that incorporated both local and global context in a joint training objective (Huang et al., 2012), as well as methods that jointly performed word sense clustering and embedding learning as in Neelakantan et al. (2014); Li and Juraf-sky (2015). A probabilistic framework was introduced by Tian et al. (2014), where the Skip-Gram model of Word2Vec was modified to learn multiple embedding vectors. Furthermore, latent topics 1 https://github.com/Elbria/utdsm_ naacl2018 2 We limit our discussion to related works that use monolingual DSMs and corpora. were integrated into the Skip-Gram model, resulting in topical word embeddings which modeled the semantics of a word under different contexts (Liu et al., 2015b,a; Nguyen et al., 2017). Another topic-related embedding creation approach was proposed in Christopoulou et al. (2018) where a mixture of topic-based semantic models was extracted by topical adaptation of in-domain corpora. Other approaches used autoencoders (Amiri et al., 2016), convolutional neural networks designed to produce context representations that reflected the order of words in a context (Zheng et al., 2017) and reinforcement learning (Lee and Chen, 2017; Guo et al., 2018). Supervised approaches, based on prior knowledge acquired by sense inventories (e.g., WordNet) along with word sense disambiguation algorithms, were also introduced for sense-specific representations extraction (Chen et al., 2014; Ia-cobacci et al., 2015). In other works, pre-trained word embeddings have been extended to embeddings of lexemes and synsets (Rothe and Schutze, 2015) or were de-conflated into their constituent sense representations (Pilehvar and Collier, 2016) by exploiting semantic lexical resources. Our system follows a four-step approach: 1. Global Distributional Semantic Model. Given a large collection of text data we train a DSM that encodes the contextual semantics of each word into a single representation, also referred to as Global-DSM. 2. Topic-based Distributional Semantic Models. Next, a topic model is trained using the same corpus. The topic model splits the corpus into K (possibly overlapping) sub-corpora. A DSM is then trained from each sub-corpus resulting in K topic-based DSMs (TDSMs). The topical adaptation of the semantic space takes into account the contextual variations a word exhibits under different thematic domains and therefore leads to the creation of topic-specific vectors (topic embeddings). an unsupervised self-learning scheme. In the unified semantic space each word is represented by a set of topic embeddings that were previously isolated in distinct vector spaces, thus creating a Unified multi-Topic DSM (UTDSM). 4. Smoothing of topic embeddings. As an optional step, we employ a smoothing approach in order to cluster a word's topic embeddings into N Gaussian distributions via a Gaussian Mixture Model (GMM). This step lessens the noise introduced to our system through the semantic mappings and sparse training data. Figure 1 : Simplified depiction summarizing the intuition behind the alignment process of topic embeddings. In the unified vector space, the polysemous word cancer is represented by two topic vectors that capture different semantic properties of the word under a zodiacal and a medical topic. Words astrology and tumor are examples of semantic anchors that define the mappings. The first step towards the thematic adaptation of the semantic space is the induction of in-domain corpora, using the Latent Dirichlet Algorithm (LDA) (Blei et al., 2003). LDA is a generative probabilistic model of a corpus. Its core idea is that documents are represented as random mixtures over topics; where each topic is defined as a probability distribution over a collection of words. Given as input a corpus of documents, LDA trains a topic model and creates a distribution of words for each topic in the corpus. Using the trained LDA model, we infer a topic distribution for each sentence in the corpus. Afterward, following a soft clustering scheme each sentence is included in a topic-specific corpus when the posterior probability for the corresponding topic exceeds a prede-fined threshold. The resulting topic sub-corpora are then used to train topic-based DSMs. Any of the DSM training algorithms proposed in the literature can be used for this purpose; in this paper, we opt for the Word2Vec model (Mikolov et al., 2013a). The intrinsic non-determinism of the Word2Vec algorithm leads to the creation of continuous vector spaces that are not naturally aligned to a unified semantic reference space, precluding the comparison between words of different thematic domains. To circumvent this limitation, we need to map the word representations of TDSMs to a shared vector space. In particular, we hypothesize that TDSMs capture meaningful variations in usage of polysemous words, while the relative semantic distance between monosemous words is preserved. This hypothesis motivated us to think of monosemous words as anchors between semantic spaces, as illustrated in Figure 1. One way to retrieve the list of anchors is to extract monosemous words from lexical resources such as WordNet (Prokhorov et al., 2017). However, this method is restricted to languages where such lexical resources exist and depends on the lexical coverage and quality of such resources. To overcome the above limitations, we propose a fully unsupervised method for semantic anchor induction. Although the embeddings of the topic and global semantic vector spaces are not aligned, their corresponding similarity matrices (once normalized) are. Based on this observation, we compute the similarity between a given word and every other word in the vocabulary (similarity distribution) for the different topic and global spaces. Then, we assume that good semantic anchors should have similar similarity distributions across the topic-specific and the global space, as illustrated in Figure 2. Artetxe et al. (2018) was based on a similar observation to align vector semantic spaces in bilingual machine translation context. Let V be the intersection of the Global-DSM and the K TDSMs vocabularies and d the embedding dimension. We then define X k  R | V | d as the embedding matrix of the k -th TDSM, and Y  R | V | d as the embedding matrix of the global DSM, where the i -th row of each matrix corresponds to the unit normalized representation of a word in V . Then, we define S k = X k X Tk , S g = Y YT  R | V || V | to be the similarity distribution matrices for the k -th TDSM and the global-DSM, respectively. Then our objective is to extract a list of semantic anchors A that minimizes the Euclidean distance between the two different similarity distributions. Specifically, for every word i we calculate the average semantic distribution across all topics: <s ik > k = 1 KK (cid:88) k =1 s ik (1) (cid:107) <s ik > k  s ig (cid:107) 2 ,  i = 1 , . . . , | V | (2) where s ig , s ik is the i -th row of the S g and S k similarity matrix, respectively, representing the similarity distribution between word i and every other word in the vocabulary V . We then choose | A | anchors as the words with the smallest values according to criterion 2. Furthermore, we assume that there exists an orthogonal transformation matrix between the topic embeddings of the extracted semantic anchors of each TDSM (source space) and the corresponding representations of the global-DSM (target space). The orthogonality constraint on the transformation matrix is widely adopted by the literature for various semantic space alignment tasks (Xing et al., 2015; Artetxe et al., 2016; Hamilton et al.). Assume  jk  R d is the vector representation of the j -th anchor word in the source space and  jg  R d is its corresponding vector representation in the target space. The transformation matrix M k  R d  d that projects the first space to the latter is learned via solving the following constraint optimization problem: 3 min M k | A | (cid:88) j =1 (cid:107) M k  jk   jg (cid:107) 22 , s.t. M k M Tk = I (3) The induction of multiple topic embeddings in the unified vector space is achieved via applying Equation 3 to each TDSM. Specifically, given a word and its k -th topic distributed representation x k  R d , we compute its projected representation x (cid:48) k  R d as follows: x (cid:48) k = M k x k (4) 3.3 Smoothing Of Topic Embeddings Starting from the set of aligned topic embeddings { x (cid:48) k } Kk =1 for each word, we learn a Gaussian Mixture Model with N components, where closely positioned topic embeddings are assigned to the same component. This step operates as an implicit way of segmenting the space of topic embeddings for each word in order to capture more useful hyper-topicsunion of topicswhich better represent their different meanings. We suggest that each Gaussian distribution forms a semantically coherent unit that corresponds to closely related semantics of the target word. Subsequently, the mean vector of each Gaussian distribution is used as a representative vector of each component, leading to a new set of smoothed topic embeddings { x  n } Nn =1 for each word, where x  n  R d . 3 This problem is known as the orthogonal Procrustes problem and it has a closed form solution as proposed in (Sch  onemann, 1966). As our initial corpus we used the English Wikipedia, containing 8 . 5 million articles (Tur-ney, 2012). During the training of the topic model, we used the articles found in the Wikipedia corpus and employed the Gensim implementation of LDA (Rubenstein and Goodenough, 1965) setting the number of topics K to 50 . Using a threshold of 0 . 1 , we followed a soft-clustering approach, to bootstrap the creation of topic sub-corpora, using our trained topic model. Finally, we used Gen-sim's implementation of Word2Vec and Continuous Bag-of-Words method to train both the global-DSM and the TDSMs. The context window parameter of Word2Vec is set to 5 , while the dimensionality d of all the constructed DSMs is equal to 300 or 500 . 4 4.2 Semantic Anchors The number of semantic anchors that determine the mappings between our source and target spaces is set to | A | = 5 000 5 according to our unsupervised approach (criterion 2). Those are selected from the common set of words that are represented in all semantic spaces with | V |  12 000 . As a second experiment, we randomly sample | A | words from the vocabulary of each TDSM to define its transformation matrix. We repeat this experiment 10 times, every time sampling a different list from the corresponding vocabulary and report average performance results. To apply the smoothing technique on the set of a word's topic embeddings we use the Scikit-learn implementation of Gaussian Mixture Model clustering algorithm (Pedregosa et al., 2011). We initialize the mean vector of each component using k-means algorithm and the parameters of the model are estimated using Expectation-Maximization (EM) algorithm. To estimate the semantic similarity between a pair of words provided in sentential context, we use 4 Any parameter not mentioned is set to default values of the corresponding implementations (e.g., Word2Vec, Gensim LDA). 5 We have experimented with different values of anchors from { 1 000 , 2 000 , 3 000 , 4 000 , 5 000 } and report results for the best setup. the standard evaluation Stanford Contextual Word Similarity (SCWS) (Huang et al., 2012) dataset which consists of 2 003 word-pairs with assigned semantic similarity scores computed as the average estimations of several human annotators. Following the evaluation guidelines proposed in literature, we employ the AvgSimC and MaxSimC contextual metrics, firstly discussed in Reisinger and Mooney (2010). In particular, given the word-pair ( w, w (cid:48) ) , and their provided contexts ( c , c (cid:48) ) we define: AvgSimC( w, w (cid:48) ) = 1 K 2 K (cid:88) j =1 K (cid:88) k =1 p( j | w, c )p( k | w (cid:48) , c (cid:48) )d( x (cid:48) j ( w ) , x (cid:48) k ( w (cid:48) )) , (5) MaxSimC( w, w (cid:48) ) = d(  x (cid:48) ( w ) ,  x (cid:48) ( w (cid:48) )) , (6) Following the notation used in 3.2, K is the number of topics returned by the trained LDA model, x (cid:48) j is the word embedding trained on the sub-corpus corresponding to the j -th topic after being projected to the unified vector space, p( j | w, c ) denotes the posterior probability of topic j returned by LDA given as input the context c of word w , d denotes the cosine similarity between the two input representations and finally  x (cid:48) ( w ) = u argmax 1  j  K p( j | w,c ) ( w ) is the vector representation of word w that corresponds to the topic with the maximum posterior for c . Intuitively, a higher score in MaxSimC indicates the existence of more robust multi-topic word representations. On the other hand, AvgSimC provides a topic-based smoothed result across different embeddings. Besides the standard evaluation benchmark of contextual word similarity, we also investigate the effectiveness of our mapped cross-topic embeddings on document and sentence level downstream NLP tasks: text classification and paraphrase identification. We report weighted-averaging precision, recall, F1-measure and accuracy performance metrics. Text classification. We used the 20 NewsGroup 6 dataset, which consists of about 20 000 documents. Our goal is to classify each document into one of the 20 different newsgroups based on its content. 6 http://qwone.com/ jason/20Newsgroups/ Paraphrase Identification. For this task we aimed at identifying whether two given sentences can be considered paraphrases or not, using the Microsoft Paraphrase dataset (Dolan et al., 2004). Document and Sentence level representations. AvgC D = 1 | D | | D | (cid:88) d =1 K (cid:88) k =1 p( k | D ) x (cid:48) k ( w d ) , (7) Avg D = 1 | D | | D | (cid:88) d =1 K (cid:88) k =1 1 K x (cid:48) k ( w d ) , (8) MaxC D = 1 | D | | D | (cid:88) w =1 x (cid:48) m ( w d ) s . t . m = argmax k =1 ,..,K { p( k | D ) } , (9) where p( k | D ) denotes the posterior probability of topic k returned by LDA given as input the sen-tence/document D and x (cid:48) k ( w d ) is the mapped representation of word w d for topic k . For the case of paraphrase identification, we extract a single feature vector for each sentence-pair via concatenating the features of the individual sentences. After feature extraction, we train a linear Support Vector Classifier (SVM) (Pedregosa et al., 2011) using the proposed train/test sets for both tasks. We report the best results for each experimental configuration after tuning the SVM's penalty parameter of the error term using 500 dimensional word embeddings. In Table 1 we compare our model (UTDSM) with our baseline (Global-DSM) and other state-of-the-art multi-prototype approaches for the contextual semantic similarity task. It is clear that all different setups of UTDSM perform better than the baseline for both contextual semantic similarity metrics. Using a single Gaussian distribution (UTDSM + GMM (1)) at the smoothing step of our method produces similar results to the baseline model. This is anticipated as both methods provide a centroid representation of a word's diverse semantics. In terms of MaxSimC the model consistently yields higher performance when the list of semantic anchors is induced via our unsupervised method instead of using randomly selected anchor words (UTDSM Random). We also observe that random anchoring performs slightly worse than UTDSM with respect to AvgSimC . This result validates our hypothesis that the representations of words, which share consistent similarity distributions across different topic domains, constitute informative semantic anchors that determine the mappings between semantic vector spaces. Furthermore, we observe that GMM smoothing has a different effect on the MaxSimC and AvgSimC metrics. Specifically, for AvgSimC we consistently report lower results when GMM smoothing is applied for different number of components. We attribute this behavior to a possible loss of model capacitydecrease in the number of topic embeddingsthat is capable of capturing additional topic information. At the same time, our smoothing technique highly improves the performance of MaxSimC for all possible configu-rations. Given that this metric is more sensitive to noisy word representations, this result indicates that our technique lessens the noise introduced to our system and captures finer-grained topic senses of words. Overall, the performance of our model is highly competitive to the state-of-the-art models in terms of AvgSimC , for 500 -dimensional topic embeddings. We also achieve state-of-the-art performance for the MaxSimC metric, using smoothed topic embeddings of 300 or 500 dimensions with 2 or 3 Gaussian components. Evaluation results on text classification are presented in Table 2. We observe that our model performs better than the baseline across all metrics for both averaging approaches ( AvgC D , Avg D ), while the usage of dominant topics appears to have lower performance ( MaxC D ). Specifically, we get an improvement of 2  2 . 5% on topic-based average and 0 . 5  1% on simple average combination compared to using Global-DSM. Results for the paraphrase identification task are presented in Table 3. Avg D yields the best results especially in F1 metric showing that cross-topic representations are semantically richer than single embeddings baseline (Global-DSM). Although we apply the topic distributions p( k | D ) extracted from LDA (document-level model) to a sentence-level task, improvements over the baseline are also shown in the AvgC D and MaxC D cases. Overall, the proposed UTDSM model outperforms the baseline Global-DSM model on both semantic similarity and downstream tasks. Finally, we carry out a cross-domain semantic analysis to detect the variations of a word's meaning in different topic domains. To that end, we use a list of known polysemous words and measure the semantic similarity between different topic representations of the same ambiguous word. The ultimate goal of this analysis is to validate that our model captures known thematic variations in semantics of polysemous words. Table 4 includes examples of our analysis. The most probable words of the topics (second column) give an intuitive sense of their major contexts, while their nearest neighbors (third column) infer the sense of the target word in the corresponding topic domain. For example, the word drug is mostly related to medication in a broad medical domain; it experiences though a slight shift from this meaning when it resides in a topic about illegal substances. Furthermore, the highly polysemous word act shifts from meaning statute to meaning performance under the corresponding law and art topics. Similar semantic variations are observed for words python , rock and nursery . Moreover, in Figure 3 we visualize the topic embeddings of seven words before and after projecting the topic-based DSMs to the unified space, using principal component analysis. We additionally depict the Gaussian distribution learned from the topic representations of each word reflecting the uncertainty of their meanings. The center of each distribution is specified by the mean vector and contour surface by the covariance matrix. On the left, we depict the position of words prior to applying the unsupervised mapping approach where the topic sub-spaces are unaligned. In the unaligned space, words demonstrate similar area coverage regardless of their polysemy. After the mappings, we see on the right that the area under a word's distribution is indicative of its degree of polysemy. Specifically, we observe that the variance of the learned representations becomes larger for the cases of polysemous words such as 7 Similar results were obtained for each metric using smoothed word embeddings. Also, there are no standard evaluation approaches for comparison of previous works on downstream tasks. 8 Note that a topic domain is described as a distribution over words in our model. python, java, adobe in order to assign some probability to their diverse meanings. Monosemous words such as snake, microsoft and malay have smaller variances. Furthermore, we observe that the semantic relationships between words are much better captured by their corresponding positions in the aligned space. We present an unsupervised approach of mapping multiple topic-based DSMs to a unified vector space in order to capture different contextual semantics of words. We assume that words having consistent similarity distributions regardless of the domain they exist in could be considered informative semantic anchors that determine the mappings between semantic spaces. The projected word embeddings yield state-of-the-art results on contextual similarity compared to previously proposed unsupervised approaches for multiple word embeddings creation, while they also outperform single vector representations in downstream NLP tasks. In addition, we provide insightful visualizations and examples that demonstrate the capability of our model to capture variations in topic semantics of words. As future work, one can hypothesize that the area a word covers in the mapped space reveals its semantic range. In this direction, a refinement of the semantic anchor selection approach could be explored in an iterative way assuming that the variance of a word's Gaussian distribution denotes its degree of polysemy (Vilnis and McCallum, 2015). Moreover, we would like to explore a more sophisticated smoothing technique where the number of Gaussian components is adapted for each word. Given that Gaussian mixture embeddings could capture the uncertainty of a word's representation in the semantic space one could also investigate different metrics for measuring the semantic relationship between word pairs that go beyond their point-wise comparison. Finally, it may be helpful to investigate non-linear mappings between semantic spaces using deep neural network architectures. Acknowledgments Thi work has been partially funded by the Baby-Robot project, supported by the EU Horizon 2020 Program undergrant # 687831 . Zellig S. Harris. 1954. Distributional structure. Word , 10(23):146162. Ignacio Iacobacci, Mohammad Taher Pilehvar, and Roberto Navigli. 2015. Sensembed: Learning sense embeddings for word and relational similarity. In Proc. Annual Meeting of the Association for Computational Linguistics (ACL) , pages 95105. Guang-He Lee and Yun-Nung Chen. 2017. Muse: Modularizing unsupervised sense embeddings. In Proc. Conference on Empirical Methods in Natural Language Processing , pages 327337. Jiwei Li and Dan Jurafsky. 2015. Do multi-sense embeddings improve natural language understanding? In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 1722 1732. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2015a. Learning context-sensitive word embeddings with neural tensor skip-gram model. In Proc. International Joint Conference on Artificial Intelligence (IJ-CAI , pages 12841290. Yang Liu, Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun. 2015b. Topical word embeddings. In Proc. AAAI Conference on Artificial Intelligence , pages 24182424. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Sch  utze. 2008. Introduction to Information Retrieval . Cambridge University Press. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. volume abs/1301.3781. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. Linguistic regularities in continuous space word representations. In HLT-NAACL . Dai Quoc Nguyen, Dat Quoc Nguyen, Ashutosh Modi, Stefan Thater, and Manfred Pinkal.
abstain objective abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain abstain abstain result objective method objective abstain other other other other other other other other other other other other method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method method abstain other method method abstain abstain other method abstain method abstain abstain method abstain abstain abstain method method abstain abstain abstain method abstain method method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain other abstain abstain other abstain abstain abstain method abstain abstain abstain abstain result abstain abstain result result result result abstain result result abstain result abstain result abstain abstain abstain abstain abstain result abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method method abstain objective abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain