In this paper, we propose a mixture model-based end-to-end method to model the syntactic-semantic dependency correlation in Semantic Role Labeling (SRL). Semantic dependencies in SRL are modeled as a distribution over semantic dependency labels conditioned on a predicate and an argument word. The semantic label distribution varies depending on Shortest Syntactic Dependency Path (SSDP) hop patterns. We target the variation of semantic label distributions using a mixture model, separately estimating semantic label distributions for different hop patterns and probabilistically clustering hop patterns with similar semantic label distributions. Experiments show that the proposed method successfully learns a cluster assignment reflecting the variation of semantic label distributions. Modeling the variation improves performance in predicting short distance semantic dependencies, in addition to the improvement on long distance semantic dependencies that previous syntax-aware methods have achieved. The proposed method achieves a small but statistically significant improvement over baseline methods in English, German, and Spanish and obtains competitive performance with state-of-the-art methods in English. 1 1 Introduction Semantic Role Labeling (SRL) answers an essential question about sentence semantics: [Who] [does what] [to whom]. A core problem of SRL is identifying semantic dependencies that specify the semantic role of arguments in relation to predicates (He et al., 2018; Kasai et al., 2019). For example, [who] (argument) is the agent (semantic role) to [does what] (predicate). Semantic dependency parsers (Dozat and Manning, 2018a) identify semantic dependencies by giving a distribution over semantic dependency labels (denoted as semantic label distribution) for all predicate-argument pairs. 1 Our code is available at this repository. In this paper, we propose a mixture model (Pear-son, 1894) based semantic dependency parser for SRL where we target the dependence of semantic label distributions on Shortest Syntactic Dependency Path (SSDP) patterns. SSDP is the shortest path connecting a predicate-argument pair in a syntactic dependency tree. Bunescu and Mooney (2005) and Cai et al. (2016) claim that SSDP encodes most information about bigram relations, such as the semantic dependency. Indeed, previous research (He et al., 2018; Xia et al., 2019) shows that modeling the correlation between SSDPs and semantic dependencies is crucial for building a high-performance SRL system. Semantic label distributions vary depending on SSDPs, even when the SSDPs connect predicate-argument pairs with the same surface words. Figure 1 shows an example where two predicate-argument pairs have different semantic dependency labels while sharing the same surface words. SSDP patterns help discriminate semantic labels between the two pairs. The example indicates the dependence of semantic label distributions on SSDP patterns. We propose a mixture model-based method (Figure 2) to model the dependence in two steps: (1) Separately estimating semantic label distributions for different SSDP patterns as component distributions, and (2) Probabilistically clustering SSDP patterns with similar semantic label distributions using a mixture weight. The mixture model estimates the semantic label distribution by aggregating the component distributions using the mixture weight. We focus on SSDP hop patterns in this paper as we observed a drastic variation in semantic label distributions for different hop patterns through the change in mutual information (Shannon et al., 1949) (Sec-tion 2). We evaluate the proposed method using the CoNLL-2009 dataset (Hajic et al., 2009) 2 , the most popular multi-lingual SRL dataset with parallel syntactic and semantic dependency annotations. Experiments show that the proposed method correctly learns a mixture weight reflecting the variation in semantic label distributions. Modeling the variation improves performance in predicting short distance semantic dependencies in addition to long distance dependencies that previous syntax-aware methods (He et al., 2018; Roth and Lapata, 2016; Strubell et al., 2018) improve only on. Previous syntax-aware methods improve their performance on long distance dependencies at the expense of the performance on short distance dependencies. In comparison, the proposed method makes no such compromise, improving its performance over semantic dependencies of all ranges. In general, the proposed method obtains a small but statistically significant improvement over baseline methods in English, German, and Spanish and achieves competitive performance with state-of-the-art methods in English. 2 While being similar, the CoNLL-2009 dependency format (Surdeanu et al., 2008) predates the well-known universal dependency format (Nivre et al., 2017) and have fundamental differences. This work follows the CoNLL-2009 format for both syntactic and semantic dependencies. in semantic label distributions for different SSDP hop patterns, (2) proposing a mixture model-based method capturing the variation, and (3) conducting a detailed experiment evaluating the proposed method. As mentioned in Section 1, SSDP affects the choice of semantic dependency labels. We study the impact of SSDP hop patterns on semantic label distributions through the change in mutual information (Shannon et al., 1949) in this section. We observe a drastic change in mutual information only for hop patterns that frequently co-occur with semantic dependencies. SSDP is the path connecting a predicate-argument pair in a syntactic dependency tree. Its hop pattern describes the number of transitions needed to transit from the predicate to the argument. We denote the hop pattern by ( ,  ) , where  is the number of dependent-to-head transitions and  is the number of head-to-dependent transitions. In a syntactic dependency tree, syntactic dependencies are arcs pointing from syntactic heads to syntactic dependents. The head-to-dependent transition moves in the same direction as the syntactic dependencies, whereas the dependent-to-head transition moves in the opposite direction. In Figure 1, the SSDP connecting eliminate and it consists of a dependent-to-head transition moving from eliminate to will, and a head-to-dependent transition moving from will to it. The hop pattern of this SSDP is (1, 1). We denote the syntactic random variable for hop patterns as X and the semantic random variable for semantic labels as Y . X maps predicate-argument 7960 word pairs ( p s , a s ) in a sentence s to their hop patterns, whereas Y maps the pairs to their semantic labels. Their mutual information MI( X, Y ) measures the reduction in uncertainty about Y after knowing X . High mutual information indicates relatively low uncertainty in the conditional distribution PY | X . To highlight the impact of hop patterns on semantic label distributions, we compare the mutual information of two ideal models, a syntax-aware model ( X ( , ) , Y ) and a syntax-agnostic model ( X 0 , Y ) . We define the syntactic variables X ( , ) and X 0 as Equation 1 and 2. This definition makes the variable X ( , ) sensitive only to the hop pattern ( ,  ) and X 0 blind to any hop pattern information. We define the mutual information gain of ( ,  ) as the difference in mutual information between the syntax-aware model and the syntax-agnostic model (Equation 3). X ( , ) ( p s , a s ) = (cid:40) 1 , ( p s , a s ) is of ( ,  ) 0 , otherwise (1) X 0 ( p s , a s ) = 0 (2) MI( X ( , ) , X 0 ) = MI( X ( , ) , Y )  MI( X 0 , Y ) (3) Figure 3 reports the mutual information gain of each hop pattern using the English training set of the CoNLL-2009 dataset. The figure shows that different hop patterns have drastically varying mutual information gains. A sharp spike of mutual information gain occurs in the hop pattern (0, 1) with a gain value of 0.149 bits, indicating a strong impact of the hop pattern (0, 1) on semantic label distributions. Hop patterns with relatively short transitions have non-zero gains ranging from 0.011 bits to 0.149 bits, which indicates the degree of impact differs drastically. These hop patterns frequently co-occur with semantic dependencies (He et al., 2018). On the other hand, hop patterns co-occurring rarely with semantic dependencies have long transitions. These hop patterns have near-zero mutual information gains in Figure 3, which indicates the weak impact of the patterns. The varying degree of impact motivates the separate estimation of semantic label distributions for different hop patterns. The amount of hop patterns with a weak impact motivates the clustering of hop patterns that share similar semantic label distributions. In this section, we present background information about syntactic and semantic dependency parsing and mixture models. We also present a brief survey about syntax-aware SRL methods using SSDP information and compare the proposed method with the previous methods. Both syntactic and semantic dependencies describe bigram relations between words, namely heads and dependents. The heads and the dependents correspond to syntactic heads and dependents in syntactic dependencies and predicates and arguments in semantic dependencies. The similarity suggests that a mechanism, such as the biaffine parser (Dozat and Manning, 2017, 2018b), can capture the two dependencies. For semantic dependencies, the biaffine parser estimates a distribution P ( r | p, a ) over relations r  R (cid:83) {  } between a predicate p and an argument a . R denotes the set of semantic relation labels, and  denotes no relation occurring between p and a . For syntactic dependencies, the biaffine parser estimates a distribution P ( h | d ) , predicting the syntactic head h of the syntactic dependent d . Neural biaffine parsers estimate the two distributions as Equation 4 3 , 5, 6 4 , and 7. e p , e a , e h and e d denote the feature vectors of p , a , h and d from a sentence encoder. r ( e p , e a ) = e Tp ( W r 1 ) e a + w r 2 ([ e p ; e a ]) + b r (4) P ( r | p, a ) = Softmax([  r ( e p , e a )]) (5)  ( e h , e d ) = e Th ( W 1 ) e d + w 2 ([ e h ; e d ]) + b (6) P ( h | d ) = Softmax([  ( e h , e d )]) (7) 3.2 Mixture Model and Latent Variable Model Mixture models assume data to be generated from a mixture distribution whose component distributions belong to the same distributional family, such as the Gaussian distributions, but possess distinct parameters. The mixture of component distributions grants additional flexibility to the mixture model. For example, the Gaussian mixture model can capture multi-mode phenomena as opposed to the simple Gaussian model (Bishop and Nasrabadi, 2007). A mixture model contains two core variables: an observable data variable 3 W r 1 is a weight matrix, w r 2 is a weight vector, and b r is a bias term for estimating the unnormalized probability score of P ( r | p, a ) . 4 Similarly, W 1 , w 2 , and b are parameters for estimating the unnormalized score of P ( h | d ) . X and a latent variable C indexing the component distribution that generates the data. The mixture model computes the marginal likelihood P  ( x ) := (cid:80) c P  ( x | c ) P  ( c ) by aggregating its component distributions P  ( x | c ) using the mixture weight P  ( c ) . The optimal parameter (i.e., the mixture weight and the parameters of component distributions) can be estimated by maximizing the log-likelihood log P  ( x ) . However, direct maximum likelihood estimation on the marginal log-likelihood is intractable for mixture models (Murphy, 2012), and the conventional Expectation-Maximization algorithm (Dempster et al., 1977) requires finding optimal parameters at each iteration. Variational Inference (Xu et al., 2015; Ba et al., 2015) maximizes a variational lowerbound of the log-likelihood (Equation 8), simultaneously optimizing the component distributions and the mixture weight. L = (cid:88) c q ( c | x ) log P  ( x | c ) P  ( c ) q ( c | x ) (8) = log P  ( x )  KL( q ( c | x ) || P  ( c | x )) (9) 3.3 Syntactic Dependency Information in Semantic Dependency Parsing Inspired by the close connection of syntactic and semantic dependencies, He et al. (2018), Roth and Lapata (2016), and Shi et al. (2020) attempt to build high-performance SRL systems using SSDP information. While the research improves performance over syntax-agnostic methods, their methods either require language-specific hyperparameters or exhibit a behavior challenging to interpret. The pruning method (He et al., 2018, 2019) is readily interpretable but requires language-specific hyperparameters. The method utilizes a statistical bias that most SSDPs rarely co-occur with semantic dependencies. It eliminates predicate-argument pairs of the infrequent SSDPs using heuristics. Whether an SSDP can co-occur with semantic dependencies is hardcoded in heuristics, making the method highly interpretable. However, the heuristics are language-specific, requiring manual tuning for every language. The neural methods (Roth and Lapata, 2016; Foland and Martin, 2015) are more language-independent but suffer from limited interpretability. The methods implicitly encode SSDP information using neural network encoders. Roth and Lapata (2016) and Foland and Martin (2015) encode SSDPs in a continuous embedding using an Long-Short Term Memory (LSTM) model or a Convolutional Neural Network model. Shi et al. (2020) jointly learns SSDP and semantic dependency information using a Transformer (Vaswani et al., 2017) by merging SSDP information with semantic dependency labels. The research reports performance improvements in one or more languages. However, interpreting the model's behavior is challenging. Neural encoders, such as the LSTM model in Roth and Lapata (2016), project SSDPs in a high-dimensional space. The high-dimensional space has a complex structure, rendering clustering analyses based on Euclidean distances less effective. Roth and Lapata (2016) interprets the behavior of their model using the clustering analysis, suggesting that their model captures many linguistic phenomena. However, the linguistic phenomena are fragmental and limited to a few syntactic constructions. In contrast, the proposed method is generic like the neural methods and interpretable like the pruning method. The proposed method optimizes its parameters using gradients of the back-propagated errors, which makes the proposed method more language-independent. As a result, the proposed method learns a mixture weight reflecting the impact of SSDP hop patterns on semantic label distributions, enabling analyses using the mixture weight. In this section, we present the proposed mixture model-based semantic dependency parser to model the dependence of semantic label distributions on SSDP hop patterns. In Section 2, we discussed the need to separately estimate semantic label distributions for different hop patterns and the need to cluster hop patterns sharing similar semantic label distributions. The proposed parser estimates semantic label distributions for different hop patterns using the component distributions and clusters hop patterns using the mixture weight of a mixture model. Figure 2 illustrates the model architecture of the proposed method. The model contains a conventional biaffine parser for syntactic dependencies and a mixture model-based biaffine parser for semantic dependencies. The syntactic parser provides a syntactic dependency tree from which the clustering component extracts hop patterns and determines the mixture weights. The biaffine parsers in 7962 the semantic parser estimate the component distributions. The semantic parser computes the semantic label distribution by aggregating the component distributions using the mixture weight. The syntactic and the semantic parser share a backbone sentence encoder, a Transformer model in our implementation. We jointly optimize the parameters of the syntactic and the semantic parser by optimizing the log-likelihood of the syntactic dependencies and a variational lowerbound of the log-likelihood (ELBo) of the semantic dependencies. We use the lowerbound as an approximation to the log-likelihood for inference because we find it works best in predicting semantic dependencies. We expand on the training objective of the semantic parser. The objective is to maximize the likelihood P  ( r | p, a ) of the observed semantic label r conditioned on the predicate p and the argument a . We rewrite the likelihood as a marginal of the joint likelihood P  ( r, c | p, a ) where c is the index of the component distributions. The joint likelihood can be decomposed as Equation 12 where the former term corresponds to the component distributions and the latter term corresponds to the mixture weight. Since we are interested in separating semantic label distributions by hop patterns, we replace the term P  ( c | p, a ) with P  ( c | ssdp( p, a )) where ssdp( p, a ) maps predicate-argument pairs to their hop patterns. P  ( c | ssdp( p, a )) also serves as the variational approximation q ( c | r, p, a ) because we assume the hop pattern, together with the predicate-argument pair, determines the semantic dependency label. This assumption removes the need to condition the component index c on the semantic label r in the variational approximation q . In this implementation, we encode hop patterns with orthogonally initialized embeddings and estimate the mixture weight of a hop pattern by applying a multi-layer perceptron followed by a softmax layer to the embedding. log P  ( r | p, a ) (10) = log (cid:88) c P  ( r, c | p, a ) (11) = log (cid:88) c P  ( r | c, p, a ) P  ( c | p, a ) (12) = log (cid:88) c P  ( r | c, p, a ) P  ( c | ssdp( p, a )) (13)  (cid:88) c P  ( c | ssdp( p, a )) log P  ( r | c, p, a ) (14) = L sem (  |X sem = ( r, p, a )) (15) Equation 16 depicts the full objective of the proposed model. It consists of a log-likelihood objective of the syntactic parser (Equation 17) and the ELBo objective of the semantic parser (Equation 15). G syn stands for the set of all syntactic dependencies ( h, d ) , whereas G sem stands for the set of all semantic dependencies ( r, p, a ) . J (  ) = (cid:88) ( h,d ) G syn L syn (  |X syn = ( h, d )) + (cid:88) ( r,p,a ) G sem L sem (  |X sem = ( r, p, a )) (16) L syn (  |X syn = ( h, d )) = log P  ( h | d ) (17) 5 Experiment In this section, we present experimental results for the proposed method. We call the proposed method as MM (mixture-model) in this section. We use the labeled attachment score (LAS) (Haji  c et al., 2009) as the primary metric. LAS is a micro-F1 score measuring how well a model recovers semantic dependencies. We conduct our experiments comparing MM with five baseline methods (Table 1) using the CoNLL-2009 dataset. We perform the comparison on all languages using the corresponding development sets. Each model will run using four randomly generated seeds to mitigate the impact of the seeds. We also compare the semantic scores (Hajic et al., 2009) of MM with state-of-the-art syntax-aware methods using the English test set. The semantic score is a micro-F1 score evaluating models' performance in the predicate identification in addition to the semantic dependency recovery. We use preidentified predicates extracted from the mate-tools (Bjrkelund et al., 2010), following the evaluation method of Roth and Lapata (2016). We evaluate MM using three word embeddings: a non-context-sensitive embedding, FastText (Joulin et al., 2016), and two context-sensitive embeddings, ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019). When comparing with state-of-the-art methods, we report results on the GloVe (Pennington et al., 2014) and the FastText embedding. However, the result on the FastText embedding is for reference only because the state-of-the-art methods report results mainly on the GloVe embedding. We use an 8-layer Transformer as the backbone encoder for MM and baseline models. We set the batch size to 5000 words, the maximum size that a P100 device can accommodate. We use the Adam optimizer (Kingma and Ba, 2015) with 7963 Method Name Syn Description Transformer No A Transformer model (Vaswani et al., 2017) using a biaffine semantic dependency parser Multitask Yes A Transformer model using two biaffine parsers for syntactic and semantic dependencies LISA Yes The Linguistically-Informed Self-Attention model (Strubell et al., 2018) PathLSTM Yes A Multitask model using dependency path embeddings (Roth and Lapata, 2016) Pruning Yes A Multitask model using the pruning technique (He et al., 2018) MM Yes A Multitask model using the proposed mixture model-based semantic dependency parser Table 1: Descriptions of baseline methods. parameters lr = 4 e  6 ,  1 = 0 . 9 , and  2 = 0 . 98 for training. We set the number of component distributions k in MM to 5 for all languages. We find that this number works for most languages in a preliminary experiment exploring k = 1 , 3 , 5 , 7 , 10 . For k > 5 , some components will not be assigned to any hop pattern, resulting in a waste of model parameters. For k < 5 , some components are forced to estimate semantic label distributions for hop patterns of different nature, resulting in a loss of performance. We do not perform back-propagation between the syntactic and the semantic parser in MM because we found the back-propagation causes negative impacts on the two parsers. We find that MM significantly improves over baseline methods on the English development set. Figure 4 reports the LAS of MM and baseline methods using box plots. MM achieves better LAS than baseline methods in all three embeddings. We conduct a series of significance tests against a null hypothesis that MM performs equally to each baseline method. The p-values of the hypothesis tests are shown in Table 2. Each cell in the table shows the p-value of a test comparing MM with a baseline method (shown in the row) on an embedding (shown in the column). The table suggests that MM significantly outperforms all baseline methods on the three embeddings, except the PathLSTM method on the FastText embedding. The significance test confirms the effectiveness of MM in modeling semantic dependencies. We find that MM learns a mixture weight reflecting the impact of hop patterns on semantic label distributions. Table 3 reports the component assignment extracted from the learned mixture weight. We extract the component assignment for hop patterns up to (5, 3). Most evidently, MM consistently assigns the hop pattern (0, 1) to a unique component in all three embeddings. This behavior agrees with our findings in Section 2 that the hop pattern has the highest mutual information gain. MM also consistently assigns hop patterns with near-zero mutual information gains to a single component. Moreover, MM clusters hop patterns with similar non-zero gains to a single component. These results suggest that semantic label distributions of different hop patterns have unique properties. MM is readily applicable for other languages beyond English using the same hyperparameter setting. Figure 5 reports the comparison of MM with the Transformer and the Multitask method on the development sets of German, Spanish, Catalan, Chinese, and Czech. The Multitask method consistently outperforms the Transformer method in all languages. In comparison, MM significantly outperforms the Multitask method in German and Spanish. MM also has an arguable improvement over the Multitask method in Catalan. In Chinese, MM performs similarly to the Multitask method but better than the Transformer method. In Czech, MM somehow fails to learn and achieves a considerably low LAS. We might need to tune the architecture or hyperparameters here, while MM stably outperforms the baseline methods in other languages. Using the Transformer method as a baseline, we find MM improves on both short and long distance semantic dependencies, whereas syntax-aware baseline methods improve only on long dis-WSJ Brown GloVe P R F1 P R F1 Zhou et al. (2020) 88.73 89.83 89.28 82.46 83.2 82.82 Li et al. (2019) 87.8 88 87.9 77 76.8 76.9 He et al. (2018) 89.7 89.3 89.5 81.9 76.9 79.3 Roth and Lapata (2016) 88.1 85.3 86.7 76.9 73.8 75.3 Kasai et al. (2019) 89 88.2 88.6 78 77.2 77.6 MM 91.03 90.13 90.58 80.59 79.21 79.83 MM (FastText) 91.16 90.19 90.71 83.93 82.64 83.28 ELMo P R F1 P R F1 Li et al. (2019) 90.5 92.1 91.3 81.7 81.9 81.8 Kasai et al. (2019) 90.3 90 90.2 81 80.5 80.8 Cai and Lapata (2019) 91.7 90.8 91.2 83.2 81.9 82.5 Lyu et al. (2019) -91 -82.2 Chen et al. (2019) -91.1 -82.7 MM 92.21 91.45 91.82 86.51 85.30 85.90 BERT P R F1 P R F1 Shi and Lin (2019)(base) 92.1 91.9 92 85.6 84.7 85.1 Shi and Lin (2019)(large) 92.4 92.3 92.4 85.7 85.8 85.7 Zhou et al. (2020) 91.21 91.19 91.2 85.65 86.09 85.87 MM 92.33 91.77 92.05 87.00 85.98 86.32 Table 4: Semantic scores of MM and state-of-the-art methods on the English test set. tance dependencies. To illustrate the finding, we group the semantic dependencies by their linear length 5 and evaluate the methods' performance on each group. We group the semantic dependencies into four bins: the short-distance bin (0-2) and the long-distance bins (3-5, 6-8, 9-inf). We then compute the relative performance score 6 of each syntax-aware method using the model with the median LAS score. Figure 6 reports the relative scores of LAS, Precision, and Recall. MM has the best relative LAS among syntax-aware methods in predict-5 l = | idx p  idx a | , where l is the linear length of the SSDP connecting the predicate p and the argument a . idx p and idx a represents the index of the predicate and the argument in the sentence. 6 s r = s syn +  s syn  , where s r represents the relative score, s syn + represents the score of the syntax-aware model, and s syn  represents the score of the syntax-agnostic Transformer model. ing short distance dependencies. On the FastText and the ELMo embedding, MM is the only method scoring a positive relative LAS (i.e., MM is the only method improving over the Transformer method). The reason is that MM achieves significantly better precision than baseline syntax-aware methods, which allows MM to overcome the lower recall. Meanwhile, MM has a performance improvement similar to the baseline syntax-aware methods in predicting long distance dependencies. MM achieves competitive performance with state-of-the-art syntax-aware methods. Table 4 reports the median semantic scores of MM and the reported scores of state-of-the-art methods on the English test set. The test set contains two sections: WSJ (in-domain) section and Brown (out-of-domain) section. MM achieves the best performance on the WSJ section on the GloVe and the ELMo embedding and performs comparably to other methods on the BERT embedding. MM also scores the best performance on the Brown section on the ELMo and the BERT embedding. We also find that MM on the FastText embedding performs better than MM on the GloVe embedding. This result is in line with a study evaluating non-context-sensitive word embeddings (Wang et al., 2019) where the FastText embedding outperforms the GloVe embedding on downstream NLP tasks. This paper presented a mixture model-based method for syntax-aware semantic dependency parsing in SRL. The method models the dependence of semantic label distributions on SSDP patterns. We focused on SSDP hop patterns because we observed a drastic variation in semantic label distributions through the change in mutual information. The proposed method successfully learned a mixture weight reflecting the variation. The method improved performance in predicting both short and long distance semantic dependencies, whereas baseline syntax-aware methods improved only on long distance dependencies. The method outperformed baseline methods by a small 7966 but statistically significant margin in many languages. Moreover, the proposed method achieved performance competitive with state-of-the-art methods in English. Nonetheless, hop patterns contain only limited information about SSDP. In the future, we plan to apply the proposed method to more informative SSDP patterns, such as labeled SSDP patterns. This research was supported by JST, CREST Grant Number JPMJCR2114, Japan.
objective abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain objective abstain result abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method other abstain abstain abstain abstain abstain abstain abstain abstain method other objective other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain method other abstain abstain other abstain method abstain method method abstain method abstain method method abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain result abstain abstain abstain abstain abstain objective other