Word embeddings learned in two languages can be mapped to a common space to produce Bilingual Word Embeddings (BWE). Unsupervised BWE methods learn such a mapping without any parallel data. However, these methods are mainly evaluated on tasks of word translation or word similarity. We show that these methods fail to capture the sentiment information and do not perform well enough on cross-lingual sentiment analysis. In this work, we propose UBiSE (Un-supervised Bilingual Sentiment Embeddings), which learns sentiment-specific word representations for two languages in a common space without any cross-lingual supervision. Our method only requires a sentiment corpus in the source language and pretrained monolingual embeddings of both languages. We evaluate our method on three language pairs for cross-lingual sentiment analysis. Experimental results show that our method outperforms previous unsupervised BWE methods and even supervised BWE methods. Our method succeeds for a distant language pair English-Basque. Lack of annotated corpora degrades the quality of sentiment analysis in low-resource languages. Cross-lingual sentiment analysis tackles this problem by adapting the sentiment resource in a resource-rich language (the source language) to a resource-poor language (the target language). Bilingual Word Embeddings (BWE) provide a way to transfer the sentiment information from the source language to the target language. There has been an increasing interest in BWE methods in re-cent years, including both supervised methods and unsupervised methods. Supervised BWE methods map the word vectors of the two languages in a common space by exploiting either a bilingual seed dictionary or other parallel data, while unsupervised BWE methods do not utilize any form of bilingual supervision. Yet, these methods are mostly evaluated on tasks of word translation or word similarity, and do not perform well enough on cross-lingual sentiment analysis as shown in Section 4. Consider the case where we want to perform sentiment analysis on the target language with merely an annotated sentiment corpus in the source language. We assume pretrained monolingual embeddings of both languages are available to us. One solution is to first align the embeddings of both languages in a common space using unsupervised BWE methods, then train a classifier based on the source sentiment corpus. In this solution, no sentiment information is utilized to learn the alignment. In this paper, we propose to exploit the sentiment information and learn sentiment-specific alignment. The sentiment information is gradually incorporated into the BWE through an iterative constraint relaxation procedure. Unlike previous work which performed alignment in a single direction by linearly mapping the source vectors to the target vector space, we propose an alignment model that maps the vectors of the two languages to a new shared space with two non-linear transformations. Our model is able to separate positive vectors from negative vectors in the bilingual space and allow such sentiment information to be transferred to the target language. Our main contributions are as follows: 1. We propose a novel approach to learn bilingual sentiment-specific word embeddings without any cross-lingual supervision and perform cross-lingual sentiment analysis with minimum resource requirement. We propose an iterative constraint relaxation procedure that gradually incorporates the sentiment information into the BWE. Our proposed approach achieves state-of-the-art results. 2. We introduce a novel sentiment-specific objective without having to explicitly build a classifier. Our approach is more explainable and better balances sentimental similarity and semantic similarity compared to previous approaches. 3. We introduce an alignment-specific objective and a simple re-normalization trick. Unlike previous BWE methods that learn orthogonal mappings, we introduce non-orthogonal mappings which enable the transfer of sentiment information from the source language to the target language. Cross-Lingual Sentiment Analysis Existing approaches for cross-lingual sentiment analysis can be mainly divided into two categories: (i) approaches that rely on machines translation (MT) systems (ii) approaches that rely on cross-lingual word embeddings. Standard MT-based approaches perform crosslingual sentiment analysis by translating the sentiment data into a selected language (e.g. En-glish). More sophisticated algorithms including co-training (Wan, 2009; Demirtas and Pech-enizkiy, 2013) and multi-view learning (Xiao and Guo, 2012) have been shown to improve performance. Zhou et al. (2015, 2016b,a) performed crosslingual sentiment analysis by learning bilingual document representations. These methods translate each document into the other language and enforce a bilingual constraint between the original document and the translated version. Bilingual Word Embeddings Word embeddings trained separately on two languages can be aligned in a shared space to produce Bilingual Word Embeddings (BWE), which support many NLP tasks including machine translation (Lam-ple et al., 2017), cross-lingual sentiment analysis (Barnes et al., 2018; Zhou et al., 2015) and crosslingual dependency parsing (Guo et al., 2015). BWE can be obtained in a supervised way using a seed dictionary (Joulin et al., 2018; Artetxe et al., 2016), or in an unsupervised way without any bilingual data. Adversarial training was the first successful attempt to learn unsupervised BWE (Zhang et al., 2017; Conneau et al., 2017). Self-learning was proposed by (Artetxe et al., 2017)to learn BWE with minimum bilingual resources, which was later extended into a fully unsupervised framework by adding an unsupervised dictionary initialization step (Artetxe et al., 2018). Multilingual Word Embeddings BWE methods can be extended to the case of multiple languages by simply mapping all the languages to the vector space of a selected language. However, directly learning multilingual word embeddings (MWE ) in a shared space has been shown to improve performance (Ammar et al., 2016; Duong et al., 2017; Chen and Cardie, 2018; Alaux et al., 2018). Yet, all these approaches are mainly evaluated on word translation and their effectiveness on cross-lingual sentiment analysis have not been empirically compared. Sentimental Embeddings Continuous word representations encode the syntactic context of a word but often ignore the information of sentiment polarity. This drawback makes them hard to distinguish words with similar syntactic context but opposite sentiment polarity (e.g. good and bad ), resulting in unsatisfactory performance on sentiment analysis. Tang et al. (2014) learned word representations that encode both syntactic context and sentiment polarity by adding an objective to classify the polarity of an n -gram. This method can be generalized to the cross-lingual setting by training monolingual sentimental embeddings on both languages then aligning them in a common space. However, it requires sentiment resources in the target language thus is impractical for low-resource languages. There are also approaches to learn sentimental embeddings in the bilingual space without any sentiment resources in the target language. Barnes et al. (2018) jointly minimized an alignment objective based on a seed dictionary, and a classification objective based on the sentiment corpus. Its performance is compared to our method in Section 4. Xu and Wan (2017) learned multilingual sentimental embeddings by extending the BiSkip model (Luong et al., 2015). However, their method does not apply to pretrained embeddings and requires large-scale parallel corpora thus is not included in our experiments. This subsection first introduces the proposed mappings for aligning the monolingual embeddings in the bilingual space, then describes the general self-learning algorithm used to learn these bilingual mappings. The details of our algorithm are explained in Section 3.2 Section 3.6. We assume we have normalized monolingual embeddings S  R v  d and T  R v  d , where the i -th row of S is the vector representation of word i in the source language. The normalization procedure is as follows: (i) l 2 -normalize each vector (ii) cen-ter the vectors (iii) l 2 -normalize each vector again (Artetxe et al., 2018). Given these monolingual embeddings, existing BWE methods typically learn a projection matrix W  R d  d from the source vector space to the target vector space. However, these methods are unsuitable in our setting for two reasons: (i) most methods constrain W to be orthogonal or near-orthogonal, thus preserving distances between word vectors; (ii) word vectors in the target language space remain unchanged. These two properties prevent us from separating words with opposite sentiment polarity in the bilingual space. In this work, we propose to align the monolingual embeddings with two non-linear mappings: f s ( x ) = W s x (cid:107) W s x (cid:107) f t ( x ) = W t x (cid:107) W t x (cid:107) where (cid:107)  (cid:107) denotes the l 2 -norm, W s ( W t ) is the projection matrix for the source (target) embeddings, and x is a d -dimension word vector. Each mapping can be seen as a linear projection followed by a re-normalization step. We propose the following convex domain D = { W  R d  d | (cid:107) W (cid:107) 2  r } as an alternative for the orthogonal constraint, where (cid:107)  (cid:107) 2 denotes the spectral norm and r is a hyperparameter that determines to what extent we want to preserve word distances. This is inspired by the unit spectral norm constraint proposed by (Joulin et al., 2018). word pairs in the dictionary to have similar representations in the bilingual space. In the unsupervised case, such a dictionary can be induced from the monolingual embeddings S and T (Artetxe et al., 2018). However, the quality of this dictionary is usually not good, which in turn degrades the quality of the projection matrices learned from this dictionary. Previous work (Artetxe et al., 2017, 2018) showed that an iterative self-learning procedure can induce a good bilingual dictionary and hence good projection matrices. Given an initial dictionary D bi , this procedure iterates over two steps: (i) it aligns the monolingual embeddings in a common space based on D bi , yielding S (cid:48) and T (cid:48) ; (ii) it computes a new dictionary D bi using nearest neighbour retrieval over the approximately aligned embeddings S (cid:48) and T (cid:48) . In our method, there are three objects W s , W t and D bi to update through the self-learning procedure. Thus we iterates over the following three steps: 1. Solve W s by minimizing a sentiment-specific objective L s over D , as described in Section 3.3; 2. Solve W t by minimizing an alignment-specific objective L t over D , as described in Section 3.4; 3. Derive a new bilingual dictionary D bi based on S (cid:48) = SW (cid:62) s and T (cid:48) = TW (cid:62) t , as described in Section 3.5. The normalized embeddings S and T are not aligned along the first axis, i.e., the i -th row of S does not correspond to the i -th row of T . Therefore, an initial bilingual dictionary is required in order to access the correspondence between the two languages. Following (Artetxe et al., 2018), we first compute the similarity matrices M s =  SS (cid:62) and M t =  TT (cid:62) , sort them along the second axis and normalize the rows, yielding M (cid:48) s and M (cid:48) t . For each row in M (cid:48) s , we apply nearest neighbour retrieval over the rows of M (cid:48) t to find its corresponding translation, yielding a dictionary D s  t = { (1 , T s  t (1)) , (2 , T s  t (2)) , . . . , ( v, T s  t ( v )) } , where T s  t ( i ) is the translation of the source word i . The same procedure is repeated in the other direction, yielding D t  s . The two dictionaries are then concatenated to produce the initial bilingual dictionary D bi = D s  t  D t  s . In order to incorporate the sentiment information into the bilingual word embeddings, we need a set of d -dimension vectors with known sentiment polarity. We propose a neural network based approach to learn these sentiment-specific vectors. Let the training corpus in the source language be C = { ( z 1 , y 1 ) , ( z 2 , y 2 ) , . . . , ( z |C| , y |C| ) } , where z i is a text and y i is its corresponding label. A d dimension vector with sentiment polarity y i can be obtained by calculating the weighted average of the word vectors in z i : h i = (cid:80) j  z i exp(  j ) S j  (cid:80) j  z i exp(  j ) (1) where S j  is the vector representation of the word j in the source language (corresponding to the j th row of S ) and  j is a scalar that scores the importance of word j on the sentiment polarity. j is computed by  j = max( AS j  + b ) , where A  R h  d and b  R h are the parameters to learn. This function can be seen as a convolution layer with h filters followed by a max pooling layer. The number of filters h is set to 4. Each h i is then forwarded to a linear classifier to predict the sentiment label y i . Once we have trained the model by minimizing the cross-entropy loss, we re-compute h i for each training example z i . We denote the set of vectors (i.e., h i ) with positive labels as P = { h p 1 , h p 2 , . . . , h p |P| } and the set of vectors with negative labels as N = { h n 1 , h n 2 , . . . , h n |N| } . In the 4-class setup, we have four sets: P , N , SP (the set of strongly positive vectors), SN (the set of strongly negative vectors). Given a set of positive d -dimension vectors P = { h p 1 , h p 2 , . . . , h p |P| } and a set of negative d dimension vectors N = { h n 1 , h n 2 , . . . , h n |N| } (or four sets in the 4-class setup), our goal is to distinguish the positive vectors from the negative vectors in the bilingual space, i.e., to separate W s h pi from W s h nj for any pair of i, j . We introduce a new d -dimension vector a p  O = { x  R d | (cid:107) x (cid:107)  1 } to represent the positive direction, which is to be learned. In order to separate positive vectors from negative vectors in the bilingual space, we try to make W s h pi ( i = 1 , . . . , |P| ) to be close to a p and W s h ni ( i = 1 , . . . , |N | ) to be distant from a p . For a given a p , we first compute a p (cid:62) W s h pi for i = 1 , 2 , . . . , |P| and denote the set of i with  |P| smallest values as Q p + , where   [0 , 1] is a hyperparameter 1 . These W s h pi are least similar with a p (dot product is used as the similarity metric), hence we maximize the average of a p (cid:62) W s h pi over Q p + . Likewise, we denote the set of i  { 1 , 2 , . . . , |N |} with  |N | largest values of a p (cid:62) W s h ni as Q p  . These W s h ni are most similar to a p , hence we minimize the average of a p (cid:62) W s h ni over Q p  . The overall objective is as follows: min W s D a p O L s ( W s , a p ) = L (cid:48) ( W s , a p , P , N )  =  1  |P| (cid:88) i Q p + a p (cid:62) W s h pi + 1  |N | (cid:88) i Q p  a p (cid:62) W s h ni (2) where D is the convex set defined in Section 3.1.1. The rationale for this objective is that, instead of forcing every W s h pi to be close to a p , we only focus on a fraction of positive vectors that are most distant from a p , and vice versa for those negative vectors. We observe that this objective can be rewritten as: min W s D a p O L s ( W s , a p ) = 1  |P| max QS  |P| ( |P| )  (cid:88) i Q a p (cid:62) W s h p i + 1  |N | max QS  |N| ( |N| ) (cid:88) i Q a p (cid:62) W s h ni (3) where S  |P| ( |P| ) represents all subsets of { 1 , 2 , . . . , |P|} of size  |P| , and S  |N| ( |N | ) is similarly defined. 2 This formulation shows that 1 For simplicity, we assume  |P| is already rounded to an integer. 2 There is no need to introduce a new vector a n to represent the negative direction and introduce a new objective, since the new objective is exactly the same after replacing a p with  a n . both terms of this objective can be seen as a maximum of linear functions of either W s or a p . Therefore, our objective is convex with respect to either W s or a p , thus can be efficiently minimized by using the projected gradient descent algorithm. We first minimize this objective with respect to a p over O , then minimize it with respect to W s over D . While this objective is useful in the binary setup, it does not separate a strongly positive vector in SP from a weakly positive vector in P (sim-ilarly for SN and N ). In order to achieve better performance in the 4-class setup, we adopt the one-versus-rest strategy to write L s as the sum of four terms: min W s D a p O a sp O a n O a sn O L s ( W s , a p , a sp , a n , a sn ) = L (cid:48) ( W s , a p , P , N  SP  SN ) + L (cid:48) ( W s , a sp , SP , P  N  SN ) + L (cid:48) ( W s , a n , N , P  SP  SN ) + L (cid:48) ( W s , a sn , SN , P  SP  N ) (4) where L (cid:48) is defined in Eq.(2) and a c is a d dimension vector representing the direction of class c . Based on the current bilingual dictionary D bi , we construct two sets of vectors { x s 1 , x s 2 , . . . , x s 2 v } and { x t 1 , x t 2 , . . . , x t 2 v } , where x si and x ti are the vector representations of the i -th word pair in D bi . With W s fixed, we can solve W t by minimizing: min W t D L t ( W t ) = 2 v (cid:88) i =1 (cid:107) W s x si  W t x ti (cid:107) 2 (5) where D is the convex set defined in Section 3.1.1. This objective is convex with respect to W t , thus can be minimized efficiently by using the projected gradient descent algorithm. Once we have computed W s and W t , we can obtain the aligned embeddings S (cid:48) = SW (cid:62) s and T (cid:48) = TW (cid:62) t . Then we induce a new dictionary D bi using nearest neighbour retrieval over the rows of S (cid:48) and T (cid:48) . We perform the induction in two directions to produce D s  t and D t  s , then concatenate them to produce D bi . In this work, we propose a modified version of CSLS(Conneau et al., 2017) to be used as the similarity metric to preform nearest neighbour retrieval: CSLS (cid:48) ( x , y ) = x (cid:62) y  1 k (cid:88) y (cid:48) N Y ( x ) x (cid:62) y (cid:48)  1 k (cid:88) x (cid:48) N X ( y ) x (cid:48)(cid:62) y (6) where NY ( x ) is the set of k nearest neighbours of x in the set of vectors Y (in our case Y is the set of rows of T (cid:48) ). We set k to 10 following the original paper. As mentioned in Section 3.1.1, we introduce a hyperparameter r to define the convex domain D . There is a trade-off to make for r : a large r better incorporates sentimental similarity but significantly harms the quality of the alignment, while a small r constrains W s to be near-orthogonal thus prevents it to capture the sentimental similarity. In order to address this problem, we propose to first set r to 1 , letting the the monolingual embeddings to be properly aligned. Then r is iteratively increased by  r , causing the positive vectors in the bilingual space to be gradually moved further away from the negative vectors. The training process stops when r reaches a maximum value r max , where r max is a hyperparameter 3 . The pseudo code of UBISE in the binary setup is shown in Algorithm 1. For the 4-class UBISE, lines 3,6,7 are replaced by their counterparts in the 4-class setup. We use the multilingual sentiment dataset provided by (Barnes et al., 2018). It contains annotated hotel reviews in English (EN), Spanish (ES), Catalan (CA) and Basque (EU). In our experiment, we use EN as the source language and ES, CA, EU as the target languages. For each target language, the dataset is divided into a target development set and a target test set. We also combine the strong and weak labels to produce a binary setup. 3 Although having the number of iterations be implicitly defined by r max and  r makes choosing a small r max impractical, it allows us to tune r max in a single training process. 1: r  1 2: Initialize W s and W t to identity matrices 3: Learn P , N from S and C , according to Section 3.2.2 4: Compute the initial bilingual dictionary D bi from S and T , according to Section 3.2.1 5: while r  r max do 6: a p  argmin a p O L s ( a p , W s ) 7: W s  argmin W s D L s ( a p , W s ) 8: W t  argmin W t D L t ( W t ) 9: S (cid:48)  SW (cid:62) s 10: T (cid:48)  TW (cid:62) t 11: Derive a new bilingual dictionary D bi from S (cid:48) and T (cid:48) , according to Section 3.5 12: r  r +  r 13: end while Normalize the rows of S (cid:48) , T (cid:48) to unit length (Bojanowski et al., 2017) are used by all methods. The MUSE dataset(Conneau et al., 2017) is used by approaches that require bilingual supervision 4 . Each dictionary contains 5000 unique source words. We empirically set  r = 0 . 01 and v = 10000 . The vocabulary of each language is limited to the v most frequent words so that the embedding matrix has shape v  d . Hyper parameters  and r max are tuned on the target development set via a grid search. We apply stochastic dictionary induction by randomly setting the elements of the similarity matrix used for nearest neighbour retrieval to zero with probability 1  p , as described in (Artetxe et al., 2018). p is initialized to 0 . 1 and increased by 0 . 005 at each iteration. We empirically stop updating the dictionary when r exceeds 3 . We compare our method with the following baselines, including state-of-the-art BWE methods that are originally evaluated on the word translation task, as well as bilingual sentimental embed-4 embed-4 This dataset does not contain a dictionary for EN-EU, thus we translate the EN-ES dictionary into EN-EU dings methods that are optimized for cross-lingual sentiment analysis. The bilingual word embeddings learned by each method are later evaluated on cross-lingual sentiment analysis using the same classifier for fairness. ADVERSARIAL Conneau et al. (2017) proposed an unsupervised BWE method based on adversarial training. After a near-orthogonal projection matrix is learned through adversarial training, a refinement procedure is applied to improve the quality of the alignment. VECMAP Artetxe et al. (2018) proposed an unsupervised BWE learning framework. It consists of an unsupervised dictionary initialization step and the self-learning procedure mentioned in Section 3.1.2. PROCRUSTES Artetxe et al. (2016) proposed a simple and effective supervised BWE method that requires a seed dictionary. It computes the optimal projection matrix by taking singular value decomposition (SVD). RCSLS Joulin et al. (2018) proposed an supervised BWE method that also requires a seed dictionary. They proposed a training objective that is consistent with the retieval criterion that can be minimized by using gradient descent. It achieves state-of-the-art results on the word translation task. BLSE Barnes et al. (2018) exploited both bilingual supervision and the sentiment corpus to learn bilingual sentimental embeddings. They jointly minimize an alignment-specific objective and a classification objective to learn the projection matrices. The trade-off between the two objectives is controlled by a hyperparameter   [0 , 1] . We tune  on the target development set as described in the original paper. Once the projection matrices have been learned, the classifier in this model is abandoned. The quality of the resulting BWE is evaluated using the classifier mentioned in Section 4.4. We use DAN (Iyyer et al., 2015) as the classifier to preform cross-lingual sentiment analysis. The loss of each instance is weighted by its inverse class frequency to address the class imbalance problem. For each method, the dropout rate is fixed at 0.3 and the l 2 -regularization strength is tuned on the target development set 5 . We train five classifiers for each method and report the average macro-F1 on the target test set. Table 1 presents the results of different BWE methods. UBISE outperforms all unsupervised methods on all six tasks and outperforms all baselines on four out of six tasks. All methods, especially unsupervised methods, suffer from distant language pairs, which is consistent with the observation of (Sgaard et al., 2018). VECMAP and ADVERSARIAL perform significantly worse on EN-EU compared to supervised methods. Yet, UBISE outperforms the strongest baseline by 2 . 1% on EN-EU, indicating that incorporating sentiment is vital for crosslingual sentiment analysis on distant languages. Despite the similar performance across different BWE methods in the binary setup, UBISE outperform all baselines in the 4-class setup by a large margin (average of +2 . 2% ). This may indicate that the original monolingual embeddings are able to distinguish positive words from negative words(e.g., good and bad ), but bad at distinguishing strongly positive words from weakly positive words (e.g., good and perfect ). The performance of BLSE is merely comparative with other baselines. 6 We suspect that this is due to the classifier we use to perform crosslingual sentiment analysis. The original paper used SVM or logistic regression to perform classification, in which case BLSE achieved better performance due to the utilization of sentiment information. But if we use a deeper neural network to perform cross-lingual sentiment analysis, preserving the original semantic similarity is more important. A qualitative comparison between BLSE and UBISE is presented in Section 4.8. We perform an ablation test to demonstrate the effect of the sentiment information provided by L s . 5 The optimal regularization strength depends on the BWE method. Stronger regularization is favourable to BLSE and UBISE. 6 We already obtain significantly better results after replacing the original classifier with DAN, compared with the original reported results. We create a new model UBISE MIN that does not utilize the sentiment information by eliminating lines 6,7,12 in Algorithm 1. UBISE MIN runs 500 iterations for every language pairs. The comparative results in Table 2 show that utilizing the sentiment information leads to an average improvement of +3 . 1% in the binary setup or +4 . 1% in the 4-class setup. Re-normalization is useful in the sense that it leads to better alignment by constraining all the bilingual vectors to be on the unit sphere. While this property does not matter for word translation as long as cosine-similarity is used as the retrieval criterion, it matters for cross-lingual sentiment analysis. Another effect of re-normalization is that it introduces non-linearity between the linear projection and the classifier, which is vital for separating words with opposite sentiment polarity. Without non-linearity the linear projection and the first layer of the classifier would collapse into a single linear projection, thus eliminating the effect of W s . Figure 2 illustrates how this nonlinearity helps separating positive words from negative words in the bilingual space. This effect is demonstrated in Section 4.8. To illustrate how UBISE transfers sentiment information from the source language to the target language, we visualize six categories of words in the bilingual space of UBISE and BLSE using t-SNE (Maaten and Hinton, 2008). As shown in Figure 1, both methods manage to separate positive words from negative words without any annotated data in Spanish. However, Barnes et al. (2018) abandon the original semantic similarity, which degrades its performance as shown in Section 4.5. In contrast, our method preserves semantic similarity by limiting the largest singular values of W s and W t to be smaller than r max . The trade-off between semantic similarity and sentimental similarity is made by choosing an appropriate r max . This paper presents a method to learn bilingual sentiment-specific word embeddings without any cross-lingual supervision. We propose a novel sentiment-specific objective that separates words Binary 4-class Bilingual Supervision Method ES CA EU ES CA EU 5 k dict. with opposite sentiment polarity in the bilingual space, and an alignment objective that enables the transfer of sentiment information from the source language to the target language. An iterative constraint relaxation procedure is applied to gradually incorporate the sentiment information into the bilingual word embeddings. We empirically evaluate our method on three language pairs for cross-lingual sentiment analysis and demonstrate its effectiveness. Experimental results show that incorporating sentiment information significantly improves the performance on fine-grained crosslingual sentiment analysis. We appreciate the anonymous reviewers for their helpful comments. Xiaojun Wan is the corresponding author.
abstain abstain abstain result objective abstain method result abstain abstain abstain abstain abstain abstain result method abstain abstain objective abstain objective abstain objective objective objective objective method objective abstain other other other other other other other other other other other other other other other other other other other other other abstain method abstain method method abstain abstain abstain abstain abstain method abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method method abstain abstain method method abstain other abstain abstain abstain method method method method method method method method method method other method method method abstain abstain method method abstain method abstain method method abstain result objective method method method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain abstain objective abstain abstain abstain