Mamoru Komachi Tokyo Metropolitan University komachi@tmu.ac.jp Abstract An event-noun is a noun that has an argument structure similar to a predicate. Recent works, including those considered state-of-the-art, ignore event-nouns or build a single model for solving both Japanese predicate argument structure analysis (PASA) and event-noun argument structure analysis (ENASA). However, because there are interactions between predicates and event-nouns, it is not sufficient to target only predicates. To address this problem, we present a multi-task learning method for PASA and ENASA. Our multitask models improved the performance of both tasks compared to a single-task model by sharing knowledge from each task. Moreover, in PASA, our models achieved state-of-the-art results in overall F1 scores on the NAIST Text Corpus. In addition, this is the first work to employ neural networks in ENASA. Japanese predicate argument structure analysis (PASA) examines semantic structures between the predicate and its arguments in a text. The identifi-cation of the argument structure such as who did what to whom? is useful for natural language processing that requires deep analysis of complicated sentences such as machine translation and recognizing textual entailment. PASA is a task targeted at predicates such as verbs and adjectives. However, there are also many nouns that have event-related arguments in a sentence. We call these nouns that refer to events event-nouns , for example, a verbal noun ( sahen nouns) such as houkoku report or a deverbal noun (nominalized forms of verbs) such as sukui rescue. Figure 1 shows examples of PASA and event-noun argument structure analysis (ENASA). In the NAIST Text Corpus (Iida et al., 2007), both predicates and event-nouns have one of three core case roles, nominative (NOM), accusative (ACC), and dative (DAT) as an argument. According to Iida et al. (2007), predicates have almost no argument in the same bunsetsu 1 phrase. However, in the case of event-nouns, approximately half of the accusative and dative arguments appear in the same bunsetsu phrase. Accordingly, although PASA and ENASA are semantically highly related, they are syntactically different tasks. However, most previous studies focused on predicates only; hence, there are few studies that focus 1 Functional chunk in Japanese. It consists of one or more content words (noun, verb, adjective, etc.) followed by zero or more function words (postposition, auxiliary verb, etc.). A verb phrase in Japanese thus cannot bear noun arguments in the same bunsetsu. on event-nouns (Komachi et al., 2007; Taira et al., 2008). To identify the semantic units of a sentence and to correctly understand syntactic relations, it is not sufficient to target only PASA. Thus, we propose a multi-task learning model that effectively leverages ENASA and improves PASA. Our proposed model is based on an end-to-end multilayer bi-directional recurrent neural network (RNN) used in recent works, and the model has networks that distinguish task-independent information and task-specific information. 1. This is the first attempt to design a multi-task learning framework for PASA and ENASA, and we show that our models improve the performance of both tasks. 2. Although our model is a simple model that does not consider the interactions between multiple predicates, it achieves a state-of-the-art result on the NAIST Text Corpus (NTC) in PASA by combining syntactic information as one of the features. 3. For ENASA, this is the first work to employ neural networks to effectively incorporate PASA. Many machine learning-based methods have been studied in Japanese PASA. Traditional models take pointwise approaches that construct independent models for each core case role (NOM, ACC, DAT). Taira et al. (2008) proposed a supervised model that learns features of each case using decision lists and support vector machines. Imamura et al. (2009) proposed a model that combines a maximum entropy model with a language model trained from large-scale newspaper articles. Hayashibe et al. (2011) designed three models exploiting argument position and type and determined the maximum likelihood output using pairwise comparison. However, the joint approach that optimizes the scores of all predicate-argument pairs in a sentence simultaneously showed better results than the pointwise approach. Yoshikawa et al. (2011) proposed a model that considers dependency between multiple predicate-argument relations using Markov logic networks. Ouchi et al. (2015) jointly optimized the combinations among multiple predicates and arguments in a sentence using a bipartite graph. Except for (Taira et al., 2008), these studies focused on the analysis of predicates while there are few studies that focus on event-nouns. Komachi et al. (2007) decomposed ENASA into two tasks: event-hood determination and argument identification; they proposed a supervised method using lexico-syntactic patterns. Event-hood determination is the most important characteristic that semantically differentiates ENASA from PASA. It is a task to determine whether a noun refers to an event (e.g., houkoku can refer to either to report or the outcome of reporting action, a report). Since the previous ENASA models adopted the pointwise approach with a single model, they did not explore the effective features in each task. In contrast, our models simultaneously optimize three core case roles. Moreover, the proposed models allow us to distinguish between task-shared and task-specific features using multi-task learning. Some neural models have achieved higher performance than traditional machine learning models in Japanese PASA. Shibata et al. (2016) replaced Ouchi et al. (2015)'s scoring function with feed forward neural networks. Matsubayashi and Inui (2017) represented a dependency path between a predicate and its argument with path embeddings and showed that even the local model without multiple predicates can outperform a global model. Moreover, some end-to-end models have been proposed in Japanese PASA. Ouchi et al. (2017) proposed an end-to-end model based on the model using eight-layer bi-directional long short-term memory (LSTM) proposed by Zhou and Xu (2015) and considered the interaction of multiple predicates simultaneously using a Grid RNN. Matsubayashi and Inui (2018) combined self-attention with Ouchi et al. (2017)'s model to directly capture interaction among multiple predicate-arguments. In particular, the model improved the performance of arguments that have no syntactic dependency with predicates and achieved a state-of-the-art result on Japanese PASA. Semantic role labeling (SRL) is a similar task to Japanese PASA. Recently, several end-to-end models using neural networks showed high performance in English SRL (Zhou and Xu, 2015; He et al., 2017; Tan et al., 2018). Strubell et al. (2018) proposed a multi-task learning model that jointly learned dependency parsing, part-of-speech tagging, predicate detection, and SRL based on multi-head self-attention. Ouchi et al. (2018) proposed a span-based SRL model using bi-directional LSTMs and achieved state-of-the-art results. The authors scored all possible spans for each label and selected correct spans satisfying constraints when decoding. In terms of the event-noun research, Gerber and Chai (2010) used pointwise mutual information (PMI) as a feature for 10 event-nouns with high frequency and iden-tified semantic roles using a logistic regression model. There were several LSTM models that also achieved high accuracy gains in Chinese SRL ( Wang et al., 2015; Roth and Lapata, 2016; Sha et al., 2016; Marcheggiani et al., 2017; Qian et al., 2017). For event-nouns, Li et al. (2009) showed that combining effective features in verbal SRL with nominal SRL can improve results. Although the authors did not demonstrate that verbal SRL also improves performance in combination with nominal SRL, we show that our model improves performance in both PASA and ENASA. Japanese predicate (event-noun) argument structure analysis is a task to extract arguments for certain predicates (event-nouns) and assign three case labels, NOM, ACC and DAT (Iida et al., 2007). Arguments are divided into four categories (Taira et al., 2008) according to the positions with their predicates (event-nouns). Inter-zero Zero anaphoric arguments and their predicate (event-noun) are not in the same sentence. A sentence w = w 1 ; w 2 ; (cid:1) (cid:1) (cid:1) ; w T and a predicate (event-noun) p = p 1 ; p 2 ; (cid:1) (cid:1) (cid:1) ; p q are given as input. Iida et al. (2006), Imamura et al. (2009), and Sasano and Kurohashi (2011) also analyze Inter-zero, which is a difficult task because the whole document must be searched. Following existing research (Ouchi et al., 2015, 2017; Matsubayashi and Inui, 2017, 2018; Taira et al., 2008), we only focus on three categories where arguments and their predicate (event-noun) are in the same sentence. In addition, we exclude the Bunsetsu category from the PASA evaluation following Ouchi et al. (2017) and Matsubayashi and Inui (2018). Our single model is based on an end-to-end approach (Zhou and Xu, 2015; Ouchi et al., 2017; Matsubayashi and Inui, 2018). Additionally, we add new features. Figure 2 shows the network architecture of our base model. Each word w t 2 [ w 1 ; (cid:1) (cid:1) (cid:1) ; w T ] is converted to a feature representation x t 2 [ x 1 ; (cid:1) (cid:1) (cid:1) ; x T ] at the input layer. We use six types of features. The feature representation x t is defined as follows: x t = x as t (cid:8) x posi t (cid:8) x dep t (cid:8) x type t (cid:8) x task p (1) where ( (cid:8) ) indicates concatenation of vectors. Argument Structure Predicate (event-noun) w p and argument candidates w t are converted to the vectors x as t 2 R 2 d w by the word embedding matrix. Position This is a feature that represents the positional relation between w p and w t . The feature is calculated by subtracting the word index of argument candidates from the word index of predicates (event-nouns). We use two types of units to represent relative position: word unit p word t and bunsetsu unit p bunsetsu t , which are converted to the word positional vector p word t 2 R d p and the bunsetsu positional vector p bunsetsu t 2 R d p , respectively, by the word and bunsetsu positional embed-3407 Figure 2: End-to-end single model. ding matrices. We concatenate these two vectors and obtain the positional vectors x posi t 2 R 2 d p . Dependency This is a feature that represents the dependency relation between w p and w t . We set five types of dependency relations: i). Argument candidates depend on the predicate (event-noun). ii). The predicate (event-noun) depends on the argument candidates. iii). No dependency relations between the predicate (event-noun) and argument candidates. iv). The predicate and candidate arguments are in the same bunsetsu. v). The event-noun and candidate arguments are in the same bunsetsu. The dependency relation d t is converted to the dependency vector x dep t 2 R d d by the dependency relation embedding matrix. The dependency type in Figure 2 shows how to make dependency features in Figure 1b as an example. We define the dependency type from the syntactic information annotated in the NTC. In previous work, dependency features are used differently from our study. Imamura et al. (2009) used a binary feature that represents whether or not there is a dependency relation between the predicate and its arguments. We employ more fine-grained relation types to adapt to event-nouns. Matsubayashi and Inui (2017) represented the interactions between a predicate and its arguments using path embedding. In contrast, we define different types for a predicate and event-noun to distinguish event-nouns from predicates and learn embeddings to find the associated latent structures. Event-hood Type This is a binary feature to flag all predicates (event-nouns) in a sentence inspired by Matsubayashi and Inui (2018). The purpose of this feature is to prevent predicates from becoming arguments and to help some event-nouns become arguments. The event-hood type vector x type t 2 R 2 of a candidate indicates [0,1] if the candidate is a predicate, [1,0] if the candidate is an event-noun, and [0,0] otherwise. The predicate and event-noun are annotated in the NTC. Task Label This is a binary feature vector x task p 2 R 1 that indicates 1 if the task is predicate argument structure analysis; otherwise, 0. We use the gated recurrent unit (GRU) ( Cho et al., 2014) for RNN. The RNN layers are made up of L layers of stacked bi-directional GRU. Additionally, we apply the residual connections (He et al., 2016) following Ouchi et al. (2017); Matsubayashi and Inui (2018). At each time step t , the hidden state h lt 2 R d h in the l 2 [1 ; (cid:1) (cid:1) (cid:1) ; L ] th layer is calculated as follows: h lt = { g l ( h l (cid:0) 1 t ; h lt (cid:0) 1 ) ( l = odd) g l ( h l (cid:0) 1 t ; h l t +1 ) ( l = even) (2) where g l ( (cid:1) ) denotes the l -th layer GRU function. In the output layer, we input each hidden state h Then, we obtain the output vector o t using the softmax function: o t = softmax( W o h Lt + b o ) (3) where W o 2 R 4 (cid:2) d h is the parameter matrix, and b o 2 R 4 is the bias term. The output vector represents the probability for each argument candidate 3408 over four labels, [NOM, ACC, DAT, ELSE]. ELSE denotes that the candidate argument does not have a case label. In testing, the maximum probability label is selected as the output label. We train the model using the cross-entropy loss function. Multi-task learning has been successfully applied to various natural language processing tasks ( Collobert et al., 2011; Sgaard and Goldberg, 2016; Luong et al., 2016; Hashimoto et al., 2017; Liu et al., 2017; Stoyanov et al., 2018; Marasovic and Frank, 2018; Strubell et al., 2018). One of the advantages of multi-task learning is that it learns better representation, which is robust against task-dependent noise by increasing training data. In this paper, we introduce multitask learning to PASA and ENASA for the first time. We propose three methods to extend the end-to-end single model to the multi-task learning model in the input layer, RNN layer, and output layer. Figure 3 shows the proposed models. Our final model combines all three methods (Figure 3e). Even if the surface form is the same, the contexts are different for predicates and event-nouns. For example, the event-noun houkoku report in Figure 1b has an argument in the same bunsetsu unlike predicates. Moreover, the event-noun also has a nominative argument role for the predicate mijikai short. Therefore, given this, we prepare a task-specific word embedding matrix that addresses the task-specific distribution of words. The predicate is converted to PASA-specific vectors x p t 2 R d  w by the PASA-specific predicate embedding matrix. Similarly, the event-noun is converted to ENASA-specific vectors x n t 2 R d  w by the ENASA-specific event-noun embedding matrix. These matrices are randomly initialized and can be learned during training. The feature vector x t is defined as follows: x t = { x t (cid:8) x p t (PASA) x t (cid:8) x n t (ENASA) (4) 4.2 Multi RNN Layer Previous work (Sgaard and Goldberg, 2016; Hashimoto et al., 2017) proposed hierarchical multi-task learning models that exploited features obtained from easy tasks for difficult tasks. These studies showed that performance improves when low-layer RNN representations are trained in easy tasks and high-layer RNN are leveraged for difficult tasks. Therefore, we construct a network that hierarchically overlaps a task-specific RNN on a task-independent RNN. Lower RNN layers learn task-independent knowledge representations. Then, the task-specific RNN adjusts the representations for each task. At each time step t , the 3409 hidden state m l  t 2 R d  h in the l  2 [1 ; (cid:1) (cid:1) (cid:1) ; L  ] -th layer is calculated as follows: m l  t = { g l  ( m l  (cid:0) 1 t ; m l  t (cid:0) 1 ) ( l  = odd) g l  ( m l  (cid:0) 1 t ; m l  t +1 ) ( l  = even) (5) g l  ( (cid:1) ) = { g l  p ( (cid:1) ) (PASA) g l  n ( (cid:1) ) (ENASA) (6) where g l  ( (cid:1) ) , g l  p ( (cid:1) ) , and g l  n ( (cid:1) ) denote the l  -th layer GRU functions. The position of arguments is different with respect to predicates and event-nouns. For example, predicates seldom have arguments in the same bunsetsu. In contrast, event-nouns often have arguments in the same bunsetsu, compound nouns, for example. Therefore, it is intuitive and natural to divide the output layer into task-independent and task-specific layers. The task-specific output vectors are calculated as follows: o p t = W po h t + b po (7) o n t = W no h t + b no (8) g t = (cid:27) ( W g h t + b g ) (9) where W po ; W no ; W g 2 R 4 (cid:2) d h are the parameter matrices, and b po ; b no ; b g 2 R 4 are the bias terms. h t is the hidden state of the last layer. We combine task-specific output vectors o p t ; o n t with task-independent output vector o t by the gate g t . c t = { g t  o t + (1 (cid:0) g t )  o p t (PASA) g t  o t + (1 (cid:0) g t )  o n t (ENASA) (10) o t = softmax( c t ) (11) where (  ) denotes the element-wise product. The output vector o t represents the probability of [NOM, ACC, DAT, ELSE]. We use NTC 1.5 for our experiments. We divide the dataset into training, development, and test sets in the same way as Taira et al. (2008). We use morphological and syntactic information, such as the word boundaries, the bunsetsu boundaries and the dependency relations provided in the NTC. case label in a sentence, we set an argument that only has a dependency relation with a predicate as a correct answer and assign the ELSE label to other arguments. If there is no dependency relation, we set an argument with the shortest distance j w p (cid:0) w t j as a correct answer. If the distance is equal, an argument on the left side of a predicate is considered a correct answer. In NTC 1.5, if there is a predicate phrase, such as verbal noun + suru , suru is annotated as a predicate word. We consider the verbal noun as the predicate word at the preprocessing step to match the surface of a predicate with that of an event-noun. Take the predicate houkoku-suru to report and an event-noun houkoku report as an example. Although w p before preprocessing are suru and houkoku , w p are unified to houkoku after preprocessing. We use pre-trained embeddings 2 for the initial values of the word embedding matrix. The initial values of the other embedding matrices are sampled according to a uniform distribution of [-0.25,0.25]. We convert words appearing more than once in the training set into word vectors and the remaining words into the unknown word vector. We adopt AdaDelta (  = 10 (cid:0) 6  (cid:26) = 0 : 95 ) as the optimization method. We set the number of epochs to 20 and evaluate the model with the highest F1 scores on the development set. Table 1 shows the hyperparameters. We evaluate each model with the NTC 1.5 test. The experimental results for the argument structure analysis of predicates and event-nouns are shown in Tables 2 and 3. 2 http://www.asahi.com/shimbun/medialab/word embedding 3410 Dep Zero Method ALL SD ALL NOM ACC DAT ALL NOM ACC DAT Ouchi+ 17 81.42 88.17 88.75 93.68 64.38 47.12 50.65 32.35 7.52 M&I 17 83.50 (cid:6) 0 : 17 89.89 91.19 95.18 61.90 51.79 54.69 41.8 17 M&I 18 83.94 (cid:6) 0 : 12 90.26 90.88 94.99 67.57 55.55 57.99 48.9 23 Single 83.62 (cid:6) 0 : 17 90.09 90.45 94.84 69.77 51.87 54.73 43.48 11.40 Multi-input 83.88 (cid:6) 0 : 11 90.27 90.65 95.12 69.86 53.01 55.82 44.68 10.77 Multi-RNN 83.91 (cid:6) 0 : 23 90.17 90.58 95.07 67.94 53.31 55.85 45.71 9.97 Multi-output 83.77 (cid:6) 0 : 20 90.13 90.68 94.89 68.16 53.93 56.73 43.79 9.45 Multi-ALL 83.82 (cid:6) 0 : 10 90.15 90.68 95.06 67.56 53.50 56.37 45.36 8.70 Multi-RNN+ DEP 84.55 (cid:6) 0 : 11 90.69 91.28 95.25 70.07 51.56 54.29 42.67 1.85 Multi-output+ DEP 84.73 (cid:6) 0 : 11 90.82 91.46 95.29 70.69 52.29 55.14 42.15 1.81 Multi-ALL+ DEP 84.75 (cid:6) 0 : 16 90.88 91.40 95.37 71.02 52.35 55.10 42.54 2.32 M&I 17 (ens. of 5) 84.07 90.24 91.59 95.29 62.61 53.66 56.47 44.7 16 M&I 18 (ens. of 10) 85.34 91.26 91.84 95.57 70.8 58.07 60.21 52.5 26 Multi-RNN+ DEP (ens. of 5) 85.85 91.61 92.11 95.87 72.63 53.41 55.96 46.10 0 Multi-output+ DEP (ens. of 5) 85.83 91.52 92.12 95.69 72.72 54.35 57.02 45.95 0 Multi-ALL+ DEP (ens. of 5) 86.01 91.63 92.15 95.80 72.95 54.99 57.84 45.20 0 Table 2: F1 scores on the PASA test set. Predicate Argument Structure Analysis The first set of rows in Table 2 shows the results of previous models. Ouchi+ 17 is the model from the Multi-Seq model in (Ouchi et al., 2017). M&I 17 is the model in (Matsubayashi and Inui, 2017). M&I 18 is the model from the MP-POOL-SELFATT model in (Matsubayashi and Inui, 2018). The second set of rows in Table 2 shows the results of the proposed models. These models do not use the dependency feature. Compared with the single model, all multi-task learning models improved the overall F1 scores. Among them, Multi-RNN improved the overall F1 score from the single model by 0.29 points. In previous work, Ouchi et al. (2017); Matsubayashi and Inui (2018) see improvements of 0.27 and 0.55 F1 points in their baseline models by considering multiple predicate-argument interactions. Therefore, we show that multi-task learning with ENASA achieved comparable effects as these studies in PASA. The third set of rows shows the results of proposed models using all features including the dependency feature. Multi-ALL+ DEP achieved the best F1 score among all the models including previous state-of-the-art models. In particular, the dependency feature was effective for Dep arguments. On the other hand, the performance for Zero arguments was poor. This result suggests that the dependency feature causes the model to optimize mainly for Dep arguments since Dep arguments are more numerous than Zero arguments. The fourth set of rows shows the results of ensemble models. Overall, our proposed model outperformed the previous ensemble model by 0.67 points in the overall F1 score. Moreover, our models are simple models that independently analyze each predicate in a sentence. Nevertheless, our models achieved higher results than Ouchi et al. (2017); Matsubayashi and Inui (2018). Although recent works have researched the method whereby multiple predicate-argument interactions are considered simultaneously, how to use syntactic in-3411 Event-noun Argument Structure Analysis The first set of rows in Table 3 shows the results of a previous model in event-noun argument structure analysis. Taira+ 08 is the model from (Taira et al., 2008). Since its scores are from NTC 1.4, the model cannot be directly compared to our models. Compared with the single model, all multi-task models improved the overall F1 scores. However, Multi-ALL+ DEP compared unfavorably with Multi-ALL even though it was the best PASA architecture. Therefore, this implies that the dependency type feature between the predicate and its argument is not effective in ENASA. In Figure 4, we compare the PASA results from test sets for each model. In Examples (a), (b) and (d), the single model failed to predict correct arguments but the Multi-RNN model correctly predicted arguments. In Example (a), the single model incorrectly predicted that arguments do not exist in this sentence. Comparing the training set of each task, although the number of event-nouns is approximately one-third of the number of predicates, the number of kessei  organize (event-nouns) is approximately twice the number of kessei  organize (predicates). Accordingly, we showed that the Multi-RNN model effectively leverages the information of event-nouns using multi-task learning. In Example (b), the single model incorrectly 3412 predicted that the NOM argument does not exist, but the multi-RNN predicted the correct arguments. Comparing the training set, there is sayuu  determine (predicate) but not sayuu   determine (event-noun). However, there are some kagi  key (arguments of predicates) in the PASA training set, and there is one kagi   key (argument of event-noun) in the ENASA training set (Example (c)). Moreover, in Example (c), dakai  break (event-noun) depends on kagi  key like sayuu  determine (predicate) in Example (b); however, no predicate depends on kagi  key in the training set. Accordingly, the Multi-RNN model also leverages the arguments of event-nouns and the positional relations between event-nouns and their arguments. Example (d) is an interesting case in which a predicate kaihi  avoid and its argument sekinin  responsibility are located in the same bunsetsu. Although this argument type (Bunsetsu) is excluded from the evaluation target in PASA, it is common as a compound noun in ENASA. Therefore, the single model wrongly predicted that the ACC argument does not exist, but multi-RNN was able to predict the answer using the specific knowledge of event-nouns. In contrast, in Example (e), the single model correctly predicted the answer, but the multi-RNN model failed to predict the correct arguments. Multi-RNN incorrectly predicted that the DAT argument does not exist in this sentence. However, ni  , a postpositional particle located after an argument, often indicates a dative case. Nevertheless, multi-RNN often predicted a wrong DAT argument by ignoring ni  . Therefore, for DAT analysis, the information of event-nouns adversely affects PASA. In Example (f), the Multi-ALL+ DEP model correctly predicted the answer, but the Multi-ALL model failed. Specifically, Multi-ALL+ DEP correctly predicted that the ACC argument is yakuwari  role, which is dependent on hatasu  play. However, the Multi-ALL incorrectly predicted that the ACC argument is kaimei  solution. Similarly, Multi-ALL without syntactic information made many mistakes, including attributive modification, such as Figure 1c. Table 4 shows the results of the two PASA models for attributive modification instances. Multi-ALL+ DEP considerably outper-ALL NOM ACC DAT Multi-ALL 80.31 83.37 72.16 19.48 Multi-ALL+ DEP 81.83 84.67 74.41 28.31 Table 4: F1 scores on the PASA test set with respect to attributive modifications. formed Multi-ALL for all cases using dependency features. Therefore, these results suggest that the dependency type feature is effective for PASA with respect to attributive modifications. We design a multi-task learning model for predicate and event-noun argument structure analysis. The experiment results show that the multi-task models outperform the single-task model on the NAIST Test Corpus for both tasks. Moreover, our model achieves a state-of-the-art result for PASA. In addition, this is the first work to employ neural networks for ENASA. In future work, we plan to consider multiple predicates and event-nouns.
abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective objective result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain other abstain abstain abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain result abstain method