An interesting and frequent type of multiword expression (MWE) is the headless MWE, for which there are no true internal syntactic dominance relations; examples include many named entities (Wells Fargo) and dates (July 5, 2020) as well as certain productive constructions (blow for blow, day after day). Despite their special status and prevalence, current dependency-annotation schemes require treating such flat structures as if they had internal syntactic heads, and most current parsers handle them in the same fashion as headed constructions. Meanwhile, outside the context of parsing, taggers are typically used for identifying MWEs, but taggers might benefit from structural information. We empirically compare these two common strategiesparsing and taggingfor predicting flat MWEs. Additionally, we propose an efficient joint decoding algorithm that combines scores from both strategies. Experimental results on the MWE-Aware English Dependency Corpus and on six non-English dependency treebanks with frequent flat structures show that: (1) tagging is more accurate than parsing for identifying flat-structure MWEs, (2) our joint decoder reconciles the two different views and, for non-BERT features, leads to higher accuracies, and (3) most of the gains result from feature sharing between the parsers and taggers. Headless multi-word expressions (MWEs), including many named entities and certain productive constructions, are frequent in natural language and are important to NLP applications. In the context of dependency-based syntactic parsing, however, they pose an interesting representational challenge. Dependency-graph formalisms for syntactic structure represent lexical items as nodes and head-dominates-modifier/argument relations between Officials at Mellon Capital were unavailable for comment O O B I O O O O nsubj case nmod mwe_NNP xcomp case nmod Figure 1: Dependency tree from the MWE-Aware English Dependency Corpus, imposing a head relationship between the words in the actually headless MWE Mellon Capital . lexical items as directed arcs on the corresponding pair of nodes. Most words can be assigned clear linguistically-motivated syntactic heads, but several frequently occurring phenomena do not easily fit into this framework, including punctuation, coordinating conjunctions, and flat, or headless MWEs. While the proper treatment of headless constructions in dependency formalisms remains debated (Kahane et al., 2017; Gerdes et al., 2018), many well-known dependency treebanks handle MWEs by giving their component words a default head, which is not indicative of a true dominance relation, but rather as a tree encoding of a flat structure without a syntactic head (de Marneffe and Nivre, 2019, pg. 213). Fig. 1 shows an example: the headless MWE Mellon Capital has its first word, Mellon , marked as the head of Capital . Despite the special status of flat structures in dependency tree annotations, most state-of-the-art dependency parsers treat all annotated relations equally, and thus do not distinguish between headed and headless constructions. When headless-span identification (e.g., as part of named-entity recognition (NER)) is the specific task at hand, begin-chunk/inside-chunk/outside-chunk (BIO) tagging (Ramshaw and Marcus, 1995) is generally adopted. It is therefore natural to ask whether parsers are as accurate as taggers in identifying these flat branches in dependency trees. Additionally, since parsing and tagging represent two different views of the same underlying structures, can joint decoding that combines scores from the two modules and/or joint training under a multitask learning (MTL) framework derive more accurate models than parsing or tagging alone? To facilitate answering these questions, we introduce a joint decoder that finds the maximum sum of scores from both BIO tagging and parsing decisions. The joint decoder incorporates a special deduction item representing continuous headless spans, while retaining the cubic-time efficiency of projective dependency parsing. The outputs are consistent structures across the tagging view and the parsing view. We perform evaluation of the different strategies on the MWE-Aware English Dependency Corpus and treebanks for five additional languages from the Universal Dependencies 2.2 corpus that have frequent multi-word headless constructions. On average, we find taggers to be more accurate than parsers at this task, providing 0 . 59% ( 1 . 42% ) absolute higher F1 scores with(out) pre-trained contextualized word representations. Our joint decoder combining jointly-trained taggers and parsers further improves the tagging strategy by 0 . 69% ( 1 . 64% ) absolute. This corroborates early evidence (Finkel and Manning, 2009) that joint modeling with parsing improves over NER. We also show that neural representation sharing through MTL is an effective strategy, as it accounts for a large portion of our observed improvements. Our code is publicly available at https://github.com/tzshi/flat-mwe-parsing . A (multi-word) headless construction, or flat structure, is a span of lexical items that together reference a single concept and where no component is a syntactically more plausible candidate for the span's head than any other component. Examples are boldfaced in the following English sentences. (1) Within the scope of this paper: a. ACL starts on July 5, 2020 . b. My bank is Wells Fargo . c. The candidates matched each other insult for insult . (Jackendoff, 2008) (1)a and (1)b show that dates and many named entities can be headless constructions, suggesting that they are frequent. Indeed, in the MWE-Aware English Dependency Corpus (Kato et al., 2017), nearly half of the sentences contain headless constructions, 75% of which are named entities. For comparison, (2) shows examples of non-flat MWEs, which are also interesting and important, but they are beyond the focus of our paper. (2) Outside the scope of this paper: a. congressman at large (Sag et al., 2002) [head = congressman] b. I have moved on . [verb-particle construction, head = moved] c. I take your argument into account . (Constant et al., 2017) [light-verb construction, head = take] Returning to headless MWEs, the choice of representation for headless spans depends on the task. In named-entity recognition , such spans are often treated as BIO tag sequences: 1 for example, in Fig. 1, Mellon is tagged as B and Capital is tagged as I. In dependency parsing , where labeled dependency arcs are the only way to express a syntactic analysis (short of treating MWEs as atomic lexical items, which would result in a chicken-and-egg problem) is to impose arcs within the MWE's span. Different corpora adopt different annotation conventions. The MWE-Aware English Dependency Corpus uses the arc label mwe_NNP , as shown in Fig. 1. The Universal Dependencies (UD; Nivre et al., 2018) annotation guidelines have all following tokens in such constructions attached to the first one via arcs labeled flat , a choice that is admittedly in principle arbitrary. 2 The frequency of flat structures across different treebanks varies according to language, genre, and even tokenization guidelines, among other factors. Table 1 lists the UD 2.2 treebanks with the highest and lowest percentage of flat relations. While the Korean treebank ko_gsd (with the highest percentage) splits up most names into multiple tokens and connects them through flat , the Japanese treebank ja_gsd (no flat s at all) treats all names as compound nouns, and thus represents them as having internal structure without any indication that a special case has occurred. 3 Fig. 2 shows examples from the UD parallel treebanks, illustrating 1 In this paper, we adopt the original BIO tagset, which cannot properly represent discontinuous MWEs. See Schneider et al. (2014) for modified tagsets providing such support. 2 universaldependencies.org/u/dep/flat.html 3 Some flat structures can end up using other dependency labels such as compound , as a result of the fact that many UD treebanks, including ja_gsd , are automatically converted from non-UD style annotations. The UD annotations depend It contains a monument to Martin Luther King , Jr. the diversity of annotation for the same sentence rendered in different languages. Overall, more than 20% of the treebanks in the UD 2.2 collection have flat structures in more than 20% of their training-set sentences. 4 Therefore, a parsing approach taking into account the special status of headless structural representations can potentially benefit models for a large number of languages and treebanks. Formally, given an n -word sentence w  w 1 , w 2 , . . . , w n , we define its dependency structure to be a graph G  p V, E q . Each node in V corresponds to a word in the sentence. Each (labeled) edge p h, m, r q P E denotes a syntactic relation labeled r between the head word w h and modifier word w m , where h, m P t 0 , 1 , . . . , n u and 0 denotes the dummy root of the sentence. Since we work with dependency treebanks, we require that the edges in E form a tree. To represent a multiword headless span w i , . . . , w j , all subsequent words in the span are attached to the beginning word w i , i.e., @ k P t i ` 1 , . . . , j u , p i, k, f q P E , where f is the special syntactic relation label de-on noting headless structures ( flat in UD annotation). Alternatively, one can also use a BIO tag sequence T  p t 1 , t 2 , . . . , t n q P t B , I , O u n to indicate the location of any headless spans within w . The headless MWE span w i , . . . , w j has the corresponding tags t i  B and @ k P t i ` 1 , . . . , j u , t k  I; tokens outside any spans are assigned the tag O. We call G and T consistent if they indicate the same set of headless spans for w . We first present the standard approaches of edge-factored parsing (3.2) and tagging (3.3) for extracting headless spans in dependency trees, and then introduce a joint decoder (3.4) that finds the global maximum among consistent (tree structure, tag sequence) pairs. Given a lengthn sentence w which we henceforth denote with the variable x for consistency with machine-learning conventionswe first extract contextualized representations from the input to associate each word with a vector x 0 (for the dummy word root), x 1 , . . . , x n . We consider two common choices of feature extractors: (1) bidirectional long short-term memory networks (bi-LSTMs; Graves and Schmidhuber, 2005) which have been widely adopted in dependency parsing (Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017) and sequence tagging (Ma and Hovy, 2016); and (2) the Transformer-based (Vaswani et al., 2017) BERT feature extractor (Devlin et al., 2019), pre-trained on large corpora and known to provide superior accuracies on both tasks (Kitaev et al., 2019; Kondratyuk and Straka, 2019). For BERT models, we fine-tune the representations from the final layer for our parsing and tagging tasks. When the BERT tokenizer renders multiple tokens from a single pre-tokenized word, we follow Kitaev et al. (2019) and use the BERT features from the last token as its representation. Since we consider headless structures that are embedded inside parse trees, it is natural to identify them through a rule-based post-processing step after full parsing. Our parsing component replicates that of the state-of-the-art Che et al. (2018) parser, which has the same parsing model as Dozat and Manning (2017). We treat unlabelled parsing as a head selection problem (Zhang et al., 2017) with deep biaffine attention scoring: h attach i  MLP attach-head p x i q m attach j  MLP attach-mod p x j q s i,j  r h attach i ; 1 s JU attach r m attach j ; 1 s P p h j  i | x q  softmax i p s : ,j q , where MLP attach-head and MLP attach-mod are multilayer perceptrons (MLPs) that project contextualized representations into a d -dimensional space; r ; 1 s indicates appending an extra entry of 1 to the vector; U att PR p d ` 1 qp d ` 1 q generates a score s i,j for w j attaching to w i (which we can then refer to as the head of w j , h j ); a softmax function de-fines a probability distribution over all syntactic head candidates in the argument vector (we use the range operator : to evoke a vector); and, recall, we represent potential heads as integers, so that we may write h j  i P t 0 , . . . , n u . The model for arc labeling employs an analogous deep biaffine scoring function: h rel i  MLP rel-head p x i q m rel j  MLP rel-mod p x j q v i,j,r  r h rel i ; 1 s JU rel r r m rel j ; 1 s P p r j  r | x, h j  i q  softmax r p v i,j, : q , where r j is the arc label between w h j and w j . The objective for training the parser is to minimize the cumulative negative log-likelihood L parse   p i  ,j  ,r  qP E r log P p h j   i  | x q  log P p r i  r  | x, h j   i  qs . After the model predicts a full parse, we extract headless structures as the tokens covered by the longest-spanning f -arcs ( f  flat in UD). 3.3 Tagging For extracting spans in texts, if one chooses to ignore the existence of parse trees, BIO tagging is a natural choice. We treat the decision for the label of each token as an individual multi-class classification problem. We let P p t i  t | x q  softmax t p MLP tag p x i qq , where MLP tag has 3 output units corresponding to the scores for tags B, I and O respectively. 5 We train the tagger to minimize L tag   i  log P p t i  t  i | x q , where t  corresponds to the gold BIO sequence. During inference, we predict the BIO tags independently at each token position and interpret the tag sequence as a set of MWE spans. As a postprocessing step, we discard all single-token spans, since the task is to predict multi-word spans. 3.4 A Joint Decoder A parser and a tagger take two different views of the same underlying data. It is thus reasonable to hypothesize that a joint decoding process that combines the scores from the two models might yield more accurate predictions. In this section, we propose such a joint decoder to find the parser+tagger-consistent structure with the highest product of probabilities. Formally, if Y is the output space for all consistent parse tree structures and BIO tag sequences, for y PY with components consisting 5 Sequence tagging is traditionally handled by conditional random fields (Lafferty et al., 2001, CRFs). However, in recent experiments using contextualized representations on tagging (Clark et al., 2018; Devlin et al., 2019), CRF-style loss functions provide little, if any, performance gains compared with simple multi-class classification solutions, at slower training speeds, to boot. Our preliminary experiments with both bi-LSTM and BERT-based encoders corroborate these findings, and thus we report results trained without CRFs. Axioms: R-INIT : i i : log P p t i  O q L-INIT : i i : 0 R-MWE : i j :  p i, j q , where  p i, j q  log P p t i  B q `  jk  i ` 1 p log P p t k  I q ` log P p h k  i qq Deduction Rules: R-COMB : i k : s 1 k j : s 2 i j : s 1 ` s 2 R-LINK : i k : s 1 k ` 1 j : s 2 i j : s 1 ` s 2 ` log P p h j  i q L-COMB : j k : s 1 k i : s 2 j i : s 1 ` s 2 L-LINK : j k  1 : s 1 k i : s 2 j i : s 1 ` s 2 ` log P p h j  i q Figure 3: Eisner's (1996) algorithm adapted to parsing headless structures (unlabeled case), our modifications highlighted in blue. All deduction items are annotated with their scores. R-MWE combines BIO tagging scores and head selection parsing scores. We need no L-MWE because of the rightward headless-structure-arc convention. of tags t i , head assignments h i , and relation labels r i , our decoder aims to find  y satisfying  y  arg max y P YP p y | x q , where P p y | x q   i P p t i | x q P p h i | x q P p r i | x, h i q . Fig. 3 illustrates our joint decoder in the unlabeled case. 6 It builds on Eisner's (1996) decoder for projective dependency parsing. In addition to having single-word spans as axioms in the deduction system, we further allow multi-word spans to enter the decoding procedures through the axiom R-MWE . Any initial single-word spans receive an O-tag score for that word, while the newly introduced MWE spans receive B-tag, I-tag, attachment and relation scores that correspond to the two consistent views of the same structure. The time complexity for this decoding algorithm remains the same O p n 3 q as the original Eisner algorithm. During training, we let the parser and the tagger share the same contextualized representation x and optimize a linearly interpolated joint objective L joint  L parse ` p 1   q L tag , 6 In the labeled case, the parser further adds the arc-labeling scores to the R-MWE and LINK rules. where  is a hyper-parameter adjusting the relative weight of each module. 7 This is an instance of multi-task learning (MTL; Caruana, 1993, 1997). MTL has proven to be a successful technique (Col-lobert and Weston, 2008) on its own; thus, in our experiments, we compare the joint decoder and using the MTL strategy alone. 4 Experiments Data We perform experiments on the MWE-Aware English Dependency Corpus (Kato et al., 2017) and treebanks selected from Universal Dependencies 2.2 (UD; Nivre et al., 2018) for having frequent occurrences of headless MWE structures. The MWE-Aware English Dependency Corpus provides automatically unified named-entity annotations based on OntoNotes 5.0 (Weischedel et al., 2013) and Stanford-style dependency trees (de Marneffe and Manning, 2008). We extract MWE spans according to mwe_NNP dependency relations. We choose the UD treebanks based on two basic properties that hold for flat structures 7 The joint decoder combines tagging and parsing scores regardless of whether the two modules are jointly trained. However, since feature extraction is the most time-consuming step in our neural models, especially with BERT-based feature extractors, it is most practical to save memory and time by sharing common feature representations across modules. Treebank # tokens # headless % # headless Average Compliance arcs spans span length ratio English 731 , 677 32 , 065 4 . 38% 16 , 997 2 . 89 100 . 00% UD 2 . 2 de_gsd 263 , 804 6 , 786 2 . 57% 5 , 663 2 . 59 93 . 00% it_postwita 99 , 441 2 , 733 2 . 75% 2 , 277 2 . 26 94 . 89% nl_alpino 186 , 046 4 , 734 2 . 54% 3 , 269 2 . 45 100 . 00% nl_lassysmall 75 , 134 4 , 408 5 . 87% 3 , 018 2 . 46 99 . 82% no_nynorsk 245 , 330 5 , 578 2 . 27% 3 , 670 2 . 54 99 . 78% pt_bosque 206 , 739 5 , 375 2 . 60% 4 , 310 2 . 25 97 . 38% Table 2: Dataset statistics. Language codes: de =German; it =Italian; nl =Dutch; no =Norwegian; pt =Portuguese. conforming to the UD annotation guidelines: (1) all words that are attached via flat relations must be leaf nodes and (2) all words within a flat span should be attached to a common head word, and each arc label should be either flat or punct . 8 For each treebank, we compute its compliance ratio , defined as the percentage of its trees containing flat arc labels that satisfy both properties above; and we filter out those with compliance ratios below 90%. 9 We rank the remaining treebanks by their ratios of flat relations among all dependency arcs, and pick those with ratios higher than 2%. Six treebanks representing 5 languages, German (McDon-ald et al., 2013), Italian (Sanguinetti et al., 2018), Dutch (Bouma and van Noord, 2017), Norwegian (Solberg et al., 2014) and Portuguese (Rademaker et al., 2017), are selected for our experiments. 10 Data statistics are given in Table 2. To construct gold-standard BIO labels, we extract MWE spans according to the longest-spanning arcs that correspond to headless structures. bi-LSTMs where each layer has 400 dimensions 8 punct inside a headless span is often used for hyphens and other internal punctuation in named entities. See the English sentence in Fig. 2 for an example. 9 The two properties defined in the UD guidelines for headless structures provide us with a common basis for uniform treatment across languages and treebanks. Unfortunately, the two properties can be violated quite often, due to issues in annotation and automatic treebank conversion into UD style. In 6 out of the top 10 treebanks containing the most flat relations, (at least one of) these properties are violated in more than 35% of the sentences with flat relations and have to be excluded from our experiments. We hope that ongoing community effort in data curation will facilitate evaluation on more diverse languages. 10 It is a coincidence that all the selected languages are Indo-European (IE). Although there are some non-IE treebanks with high flat ratio, such as Korean (see Table 1), the annotated structures frequently break one or both of the basic properties. See Fig. 2 for violation examples. in both directions and the inputs are concatenations of 100 -dimensional randomly-initialized word embeddings with the final hidden vectors of 256 -dimensional single-layer character-based bi-LSTMs; for BERT, we use pre-trained cased multi-lingual BERT models 11 and fine-tune the weights. We adopt the parameter settings of Dozat and Manning (2017) and use 500 and 100 dimensions for U att and U rel r , respectively. The MLP in the taggers have 500 hidden dimensions. We use a dropout (Srivastava et al., 2014) rate of 0 . 33 , a single hidden layer, and a ReLU activation function (Nair and Hinton, 2010) for all MLPs. The models are trained with the Adam optimizer (Kingma and Ba, 2015) using a batch size of 16 sentences. The learning rates are set to 1 e  3 for bi-LSTMs and 1 e  5 for BERT initially and then multiplied by a factor of 0 . 1 if the performance on the development set stops improving within 3200 training iterations. For the parsing models, we use the projective Eisner (1996) decoder algorithm. For the joint training and joint decoding models, we tune  P t 0 . 02 , 0 . 05 , 0 . 1 , 0 . 3 , 0 . 5 , 0 . 9 u for each treebank independently and fix the settings based on the best dev-set scores. We run each model with 5 different random seeds and report the mean and standard deviation for each setting. Results We report F1 scores based on multi-word headless-structure extraction. Table 3 compares different strategies for identifying headless MWEs in parse trees. Tagging is consistently better than parsing except for two treebanks with BERT feature extractor. Tagging beats parsing in all but two combinations of treebank and feature extractor. As hypothesized, our joint decoder improves over both strategies by 0 . 69% ( 1 . 64% ) absolute through combined decisions from parsing and tagging with(out) 11 https://github.com/huggingface/transformers w/ bi-LSTM Compl. BERT. We also compare the joint decoding setting with MTL training strategy alone. While joint decoding yields superior F1 scores, MTL is responsible for a large portion of the gains: it accounts for over half of the average gains with bi-LSTMs, and when we use pre-trained BERT feature extractors, the accuracies of jointly-trained taggers are essentially as good as joint decoding models. Interestingly, the choice of feature extractors also has an effect on the performance gap between parsers and taggers. With bi-LSTMs, tagging is 1 . 42% absolute F1 higher than parsing, and the gap is mitigated through MTL. While pre-trained BERT reduces the performance difference dramatically down to 0 . 59% absolute, MTL no longer helps parsers overcome this gap. Additionally, we observe that MTL helps both parsing and tagging models, demonstrating that the two views of the same underlying structures are complementary to each other and that learning both can be beneficial to model training. By resolving such representational discrepancies, joint decoding exhibits further accuracy improvement. In terms of dependency parsing accuracies, we confirm that our parsing-only models achieve state-of-the-art performance on the UD treebanks, but there are no significant differences in parsing results among parsing-only, MTL and jointly-decoded models. See Appendix for detailed results. Syntactic analysis in conjunction with MWE identification is an important line of research (Wehrli, 2000). The span-based representations that form the basis of phrase-structure trees (as opposed to dependency trees) are arguably directly compatible with headless spans. This motivates approaches using joint constituency-tree representations based on context-free grammars (Arun and Keller, 2005; Constant et al., 2013) and tree substitution grammars (Green et al., 2011, 2013). Finkel and Manning (2009) add new phrasal nodes to denote named entities, enabling statistical parsers trained on this modified representation to produce both parse trees and named entity spans simultaneously. Le Roux et al. (2014) use dual decomposition to develop a joint system that combines phrase-structure parsers and taggers for compound recognition. These approaches do not directly transfer to dependency-based representations since dependency trees do not explicitly represent phrases. In the context of dependency parsing, Eryigit et al. (2011) report that MWE annotations have a large impact on parsing. They find that the dependency parsers are more accurate when MWE spans are not unified into single lexical items. Similar to the phrase-structure case, Candito and Constant (2014) consider MWE identification as a side product of dependency parsing into joint representations. This parse-then-extract strategy is widely adopted (Vincze et al., 2013; Nasr et al., 2015; Simk et al., 2017). Waszczuk et al. (2019) introduce additional parameterized scoring functions for the arc labelers and use global decoding to produce consistent structures during arc-labeling steps once unlabeled dependency parse trees are predicted. Our work additionally proposes a joint decoder that combines the scores from both parsers and taggers. Alternative approaches to graph-based joint parsing and MWE identification include transition-based (Constant and Nivre, 2016) and easy-first (Constant et al., 2016) dependency parsing. These approaches typically rely on greedy decoding, whereas our joint decoder finds the globally optimal solution through dynamic programming. Our work only focuses on a subset of MWEs that do not have internal structures. There is substantial research interest in the broad area of MWEs (Sag et al., 2002; Constant et al., 2017) including recent releases of datasets (Schneider and Smith, 2015), editions of shared tasks (Savary et al., 2017; Ramisch et al., 2018) and workshops (Savary et al., 2018, 2019). We leave it to future work to extend the comparison and combination of taggers and dependency parsers to other MWE constructions. Our paper provides an empirical comparison of different strategies for extracting headless MWEs from dependency parse trees: parsing, tagging, and joint modeling. Experiments on the MWE-Aware English Dependency Corpus and UD 2.2 across five languages show that tagging, a widely-used methodology for extracting spans from texts, is more accurate than parsing for this task. When using bi-LSTM (but not BERT) representations, our proposed joint decoder reaches higher F1 scores than either of the two other strategies, by combining scores of the two different and complementary representations of the same structures. We also show that most of the gains stem from a multi-task learning strategy that shares common neural representations between the parsers and the taggers. An interesting additional use-case for our joint decoder is when a downstream task, e.g., relation extraction, requires output structures from both a parser and a tagger. Our joint decoder can find the highest-scoring consistent structures among all candidates, and thus has the potential to provide simpler model designs in downstream applications. Our study has been limited to a few treebanks in UD partially due to large variations and inconsistencies across different treebanks. Future community efforts on a unified representation of flat structures for all languages would facilitate further research on linguistically-motivated treatments of headless structures in headful dependency treebanks. Another limitation of our current work is that our joint decoder only produces projective dependency parse trees. To handle non-projectivity, one possible solution is pseudo-projective parsing (Nivre and Nilsson, 2005). We leave it to future work to design a non-projective decoder for joint parsing and headless structure extraction. We thank the three anonymous reviewers for their comments, and Igor Malioutov, Ana Smith and the Cornell NLP group for discussion and comments. TS was supported by a Bloomberg Data Science Ph.D. Fellowship.
abstain abstain abstain method objective result abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain method result abstain result abstain abstain result other other other abstain other other other other other abstain abstain other other other other other other other other other other other other other abstain other other other other other other method method method abstain method abstain method method abstain other method abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other other other other other objective other abstain method other objective method abstain objective result abstain result method abstain method abstain method other other other