When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities. If we humans cannot figure out the meaning of those expressions from the immediate local context, we consult dictionaries for definitions or search documents or the web to find other global context to help in interpretation. Can machines help us do this work? Which type of context is more important for machines to solve the problem? To answer these questions, we undertake a task of describing a given phrase in natural language based on its local and global contexts. To solve this task, we propose a neural description model that consists of two context encoders and a description decoder. In contrast to the existing methods for non-standard English explanation (Ni and Wang, 2017) and definition generation (Noraset et al., 2017; Gadetsky et al., 2018), our model appropriately takes important clues from both local and global contexts. Experimental results on three existing datasets (including WordNet, Oxford and Urban Dictionaries) and a dataset newly created from Wikipedia demonstrate the effectiveness of our method over previous work. When we read news text with emerging entities, text in unfamiliar domains, or text in foreign languages, we often encounter expressions (words or phrases) whose senses we do not understand. In such cases, we may first try to figure out the meanings of those expressions by reading the surrounding words ( local context) carefully. Failing to do so, we may consult dictionaries, and in the case of polysemous words, choose an appropriate meaning based on the context. Learning novel word senses via dictionary definitions is known to be Figure 1: Lo cal & G lobal C ontexta ware D escription generator ( LOG-CaD ). more effective than contextual guessing (Fraser, 1998; Chen, 2012). However, very often, handcrafted dictionaries do not contain definitions of expressions that are rarely used or newly created. Ultimately, we may need to read through the entire document or even search the web to find other oc-curances of the expression ( global context) so that we can guess its meaning. Can machines help us do this work? Ni and Wang (2017) have proposed a task of generating a definition for a phrase given its local context. However, they follow the strict assumption that the target phrase is newly emerged and there is only a single local context available for the phrase, which makes the task of generating an accurate and coherent definition difficult (perhaps as difficult as a human comprehending the phrase itself). On the other hand, Noraset et al. (2017) attempted to generate a definition of a word from an embedding induced from massive text (which can be seen as global context). This is followed by Gadetsky et al. (2018) that refers to a local context to disambiguate polysemous words by choosing relevant dimensions of their word embeddings. Although these research efforts revealed that both local and global contexts are useful in generating definitions, none of these studies exploited both contexts directly to describe unknown phrases. In this study, we tackle the task of describing (defining) a phrase when given its local and global contexts. We present LOG-CaD , a neural description generator (Figure 1) to directly solve this task. Given an unknown phrase without sense definitions, our model obtains a phrase embedding as its global context by composing word embeddings while also encoding the local context. The model therefore combines both pieces of information to generate a natural language description. Considering various applications where we need definitions of expressions, we evaluated our method with four datasets including WordNet (Noraset et al., 2017) for general words, the Oxford dictionary (Gadetsky et al., 2018) for polysemous words, Urban Dictionary (Ni and Wang, 2017) for rare idioms or slang, and a newly-created Wikipedia dataset for entities. Our contributions are as follows:  We propose a general task of defining unknown phrases given their contexts . We propose a method for generating natural language descriptions for unknown phrases with local and global contexts (  3). As a benchmark to evaluate the ability of the models to describe entities, we build a large-scale dataset from Wikipedia and Wikidata for the proposed task. We release our dataset and the code 1 to promote the reproducibility of the experiments (  4). The proposed method achieves the state-of-the-art performance on our new dataset and the three existing datasets used in the related studies (Noraset et al., 2017; Ni and Wang, 2017; Gadetsky et al., 2018) (  5). This task is a generalization of three related tasks (Noraset et al., 2017; Ni and Wang, 2017; Gadetsky et al., 2018) and involves various situations where we need definitions of unknown phrases (  2). In this section, we define our task of describing a phrase in a specific context. Given an unde-fined phrase X trg = { x j ,    , x k } with its context X = { x 1 ,    , x I } ( 1  j  k  I ), our task is to output a description Y = { y 1 ,    , y T } . Here, X trg can be a word or a short phrase and is included in X . Y is a definition-like concrete and concise sentence that describes the X trg . For example, given a phrase sonic boom with its context the shock wave may be caused by sonic boom or by explosion, the task is to generate a description such as sound created by an object moving fast. If the given context has been changed to this is the first official tour to support the band's latest studio effort, 2009's Sonic Boom , then the appropriate output would be al-bum by Kiss. The process of description generation can be modeled with a conditional language model as p ( Y | X, X trg ) = T (cid:89) t =1 p ( y t | y <t , X, X trg ) . (1) 3 LOG-CaD: Local & Global Context-aware Description Generator In this section, we describe our idea of utilizing local and global contexts in the description generation task, and present the details of our model. When we find an unfamiliar phrase in text and it is not defined in dictionaries, how can we humans come up with its meaning? As discussed in Section 1, we may first try to figure out the meaning of the phrase from the immediate context, and then read through the entire document or search the web to understand implicit information behind the text. In this paper, we refer to the explicit contextual information included in a given sentence with the target phrase (i.e., the X in Eq. (1)) as local con-text, and the implicit contextual information in massive text as global context. While both local and global contexts are crucial for humans to understand unfamiliar phrases, are they also useful for machines to generate descriptions? To verify this idea, we propose to incorporate both local and global contexts to describe an unknown phrase. Figure 1 shows an illustration of our LOG-CaD model. Similarly to the standard encoder-decoder model with attention (Bahdanau et al., 2015; Luong and Manning, 2016), it has a context encoder and a description decoder. The challenge here is that the decoder needs to be conditioned not only on the local context, but also on its global context. To incorporate the different types of contexts, we propose to use a gate function similar to Noraset et al. (2017) to dynamically control how the global and local contexts influence the description. Local & global context encoders We first describe how to model local and global contexts. Given a sentence X and a phrase X trg , a bidirectional LSTM (Gers et al., 1999) encoder generates a sequence of continuous vectors H = { h 1    , h I } as h i = Bi-LSTM ( h i  1 , h i +1 , x i ) , (2) where x i is the word embedding of word x i . In addition to the local context, we also utilize the global context obtained from massive text. This can be achieved by feeding a phrase embedding x trg to initialize the decoder (Noraset et al., 2017) as y 0 = x trg . Here, the phrase embedding x trg is calculated by simply summing up all the embeddings of words that consistute the phrase X trg . Note that we use a randomly-initialized vector if no pre-trained embedding is available for the words in X trg . Description decoder Using the local and global contexts, a description decoder computes the conditional probability of a description Y with Eq. (1), which can be approximated with another LSTM as s t = LSTM ( y t  1 , s (cid:48) t  1 ) , (4) d t = ATTENTION ( H , s t ) , (5) c trg = CNN ( X trg ) , (6) s (cid:48) t = GATE ( s t , x trg , c trg , d t ) , (7) p ( y t | y <t , X trg ) = softmax ( W s (cid:48) s (cid:48) t + b s (cid:48) ) , (8) where s t is a hidden state of the decoder LSTM ( s 0 = (cid:126) 0 ), and y t  1 is a jointly-trained word embedding of the previous output word y t  1 . In what follows, we explain each equation in detail. Attention on local context Considering the fact that the local context can be relatively long (e.g., around 20 words on average in our Wikipedia dataset introduced in Section 4), it is hard for the decoder to focus on important words in local contexts. In order to deal with this problem, the ATTENTION (  ) function in Eq. (5) decides which words in the local context X to focus on at each time step. d t is computed with an attention mechanism (Luong and Manning, 2016) as d t = T (cid:88) i =1  i h i , (9)  i = softmax ( U h h T i U s s t ) , (10) where U h and U s are matrices that map the encoder and decoder hidden states into a common space, respectively. Use of character information In order to capture the surface information of X trg , we construct character-level CNN s (Eq. (6)) following (No-raset et al., 2017). Note that the input to the CNN s is a sequence of words in X trg , which are concatenated with special character  , such as sonic boom. Following Noraset et al. (2017), we set the CNN kernels of length 2-6 and size 10 , 30 , 40 , 40 , 40 respectively with a stride of 1 to obtain a 160-dimensional vector c trg . Gate function to control local & global contexts In order to capture the interaction between the local and global contexts, we adopt a GATE (  ) function (Eq. (7)) which is similar to Noraset et al. (2017). The GATE (  ) function updates the LSTM output s t to s (cid:48) t depending on the global context x trg , local context d t , and character-level information c trg as f t = [ x trg ; d t ; c trg ] (11) z t =  ( W z [ f t ; s t ] + b z ) , (12) r t =  ( W r [ f t ; s t ] + b r ) , (13)  s t = tanh ( W s [( r t (cid:12) f t ); s t ] + b s ) , (14) s (cid:48) t = (1  z t ) (cid:12) s t + z t (cid:12)  s t , (15) where  (  ) , (cid:12) and ; denote the sigmoid function, element-wise multiplication, and vector concatenation, respectively. W  and b  are weight matrices and bias terms, respectively. Here, the update gate z t controls how much the original hidden state s t is to be changed, and the reset gate r t controls how much the information from f t contributes to word generation at each time step. Our goal is to let machines describe unfamiliar words and phrases, such as polysemous words, rarely used idioms, or emerging entities. Among the three existing datasets, WordNet and Oxford dictionary mainly target the words but not phrases, thus are not perfect test beds for this goal. On the other hand, although the Urban Dictionary dataset contains descriptions of rarely-used phrases, the domain of its targeted words and phrases is limited to Internet slang. In order to confirm that our model can generate the description of entities as well as polysemous words and slang, we constructed a new dataset for context-aware phrase description generation from Wikipedia 2 and Wikidata 3 which contain a wide variety of entity descriptions with contexts. The overview of the data extraction process is shown in Figure 2. Each entry in the dataset consists of (1) a phrase, (2) its description, and (3) context (a sentence). For preprocessing, we applied Stanford Tok-enizer 4 to the descriptions of Wikidata items and the articles in Wikipedia. Next, we removed phrases in parentheses from the Wikipedia articles, since they tend to be paraphrasing in other languages and work as noise. To obtain the contexts of each item in Wikidata, we extracted the 2 https://dumps.wikimedia.org/enwiki/ 20170720/ 3 https://dumps.wikimedia.org/ wikidatawiki/entities/20170802/ 4 https://nlp.stanford.edu/software/ tokenizer.shtml sentence which has a link referring to the item through all the first paragraphs of Wikipedia articles and replaced the phrase of the links with a special token [TRG] . Wikidata items with no description or no contexts are ignored. This utilization of links makes it possible to resolve the ambiguity of words and phrases in a sentence without human annotations, which is a major advantage of using Wikipedia. Note that we used only links whose anchor texts are identical to the title of the Wikipedia articles, since the users of Wikipedia sometimes link mentions to related articles. We evaluate our method by applying it to describe words in WordNet 5 (Miller, 1995) and Oxford Dictionary, 6 phrases in Urban Dictionary 7 and Wikipedia/Wikidata. 8 For all of these datasets, a given word or phrase has an inventory of senses with corresponding definitions and usage examples. These definitions are regarded as ground-truth descriptions. Datasets To evaluate our model on the word description task on WordNet, we followed Noraset et al. (2017) and extracted data from WordNet using the dict-definition 9 toolkit. Each entry in the data consists of three elements: (1) a word, (2) its definition, and (3) a usage example of the 5 https://wordnet.princeton.edu/ 6 https://en.oxforddictionaries.com/ 7 https://www.urbandictionary.com/ 8 https://www.wikidata.org 9 https://github.com/NorThanapon/ dict-definition Corpus #Phrases #Entries Phrase Context Desc. word. We split this dataset to obtain Train, Validation, and Test sets. If a word has multiple defini-tions/examples, we treat them as different entries. Note that the words are mutually exclusive across the three sets. The only difference between our dataset and theirs is that we extract the tuples only if the words have their usage examples in WordNet. Since not all entries in WordNet have usage examples, our dataset is a small subset of Noraset et al. (2017). In addition to WordNet, we use the Oxford Dictionary following Gadetsky et al. (2018), the Urban Dictionary following Ni and Wang (2017) and our Wikipedia dataset described in the previous section. Table 1 and Table 2 show the properties and statistics of the four datasets, respectively. To simulate a situation in a real application where we might not have access to global context for the target phrases, we did not train domain-specific word embeddings on each dataset. Instead, for all of the four datasets, we use the same Global Local I-Attn. pre-trained CBOW 10 vectors trained on Google news corpus as global context following previous work (Noraset et al., 2017; Gadetsky et al., 2018). If the expression to be described consists of multiple words, its phrase embedding is calculated by simply summing up all the CBOW vectors of words in the phrase, such as sonic and boom. (See Figure 1). If pre-trained CBOW embeddings are unavailable, we instead use a special [UNK] vector (which is randomly initialized with a uniform distribution) as word embeddings. Note that our pre-trained embeddings only cover 26 . 79% of the words in the expressions to be described in our Wikipedia dataset, while it covers all words in WordNet dataset (See Table 2). Even if no reliable word embeddings are available, all models can capture the character information through character-level CNN s (See Figure 1). Models We implemented four methods: (1) Global (Noraset et al., 2017), (2) Local (Ni and Wang, 2017) with CNN , (3) I-Attention (Gadetsky et al., 2018), and our proposed model, (4) LOG-CaD . The Global model is our reimplementation of the best model (S + G + CH) in Noraset et al. (2017). It can access the global context of a phrase to be described, but has no ability to read the local context. The Local model is the reimplementation of the best model (dual encoder) in Ni and Wang(2017). In order to make a fair comparison of the effectiveness of local and global contexts, we slightly modify the original implementation by Ni and Wang(2017); as the character-level encoder in the Local model, we adopt CNN s that are exactly the same as the other two models instead of the original LSTM s. The I-Attention is our reimplementation of the best model (S + I-Attention) in Gadetsky 10 GoogleNews-vectors-negative300.bin.gz at https://code.google.com/archive/p/ word2vec/ Model WordNet Oxford Urban Wikipedia Global 24.10 15.05 6.05 44.77 Local 22.34 17.90 9.03 52.94 I-Attention 23.77 17.25 10.40 44.71 LOG-CaD 24.79 18.53 10.55 53.85 Table 4: BLEU scores on four datasets. et al.(2018). Similar to our model, it uses both local and global contexts. Unlike our model, however, it does not use character information to predict descriptions. Also, it cannot directly use the local context to predict the words in descriptions. This is because the I-Attention model indirectly uses the local context only to disambiguate the phrase embedding x trg as x (cid:48) trg = x trg (cid:12) m , (16) m =  ( W m (cid:80) Ii =1 FFNN ( h i ) I + b m ) . Here, the FFNN (  ) function is a feed-forward neural network that maps the encoded local contexts h i to another space. The mapped local contexts are then averaged over the length of the sentence X to obtain a representation of the local context. This is followed by a linear layer and a sigmoid function to obtain the soft binary mask m which can filter out the unrelated information included in global context. Finally, the disambiguated phrase embedding x (cid:48) trg is then used to update the decoder hidden state as s t = LSTM ([ y t  1 ; x (cid:48) trg ] , s t  1 ) . Automatic Evaluation Table 4 shows the BLEU (Papineni et al., 2002) scores of the output descriptions. We can see that the LOG-CaD model consistently outperforms the three baselines in all four datasets. This result indicates that using both local and global contexts helps describe the unknown words/phrases correctly. While the 11 http://pytorch.org/ Input: waste Context: #1 #2 if the effort brings no compensating gain it is a waste We waste the dirty water by channeling it into the sewer Reference: useless or profitless activity to get rid of Global: to give a liquid for a liquid Local: a state of being assigned to a particular purpose to make a break of a wooden instrument I-Attention: a person who makes something that can be be be done to remove or remove the contents of LOG-CaD: a source of something that is done or done to remove a liquid Table 6: Descriptions for a word in WordNet. I-Attention model also uses local and global contexts, its performance was always lower than the LOG-CaD model. This result shows that using local context to predict description is more effective than using it to disambiguate the meanings in global context. In particular, the low BLEU scores of Global and I-Attention models on Wikipedia dataset suggest that it is necessary to learn to ignore the noisy information in global context if the coverage of pre-trained word embeddings is extremely low (see the third and fourth rows in Table 2). We suspect that the Urban Dictionary task is too difficult and the results are unreliable considering its extremely low BLEU scores and high ratio of unknown tokens in generated descriptions. Manual Evaluation To compare the proposed model and the strongest baseline in Table 4 (i.e., the Local model), we performed a human evaluation on our dataset. We randomly selected 100 samples from the test set of the Wikipedia dataset and asked three native English speakers to rate the output descriptions from 1 to 5 points as: 1) completely wrong or self-definition, 2) correct topic with wrong information, 3) correct but incomplete, 4) small details missing, 5) correct. The averaged scores are reported in Table 5. Pair-wise bootstrap resampling test (Koehn, 2004) for the annotated scores has shown that the superiority of LOG-CaD over the Local model is statistically significant ( p < 0 . 01 ). Qualitative Analysis Table 6 shows a word in the WordNet, while Table 7 and Table 8 show the examples of the entities in Wikipedia as examples. When comparing the two datasets, the quality of generated descriptions of Wikipedia dataset is sig-nificantly better than that of WordNet dataset. The main reason for this result is that the size of training data of the Wikipedia dataset is 64x larger than the WordNet dataset (See Table 1). For all examples in the three tables, the Global model can only generate a single description for each input word/phrase because it cannot access any local context. In the Wordnet dataset, only the I-Attention and LOG-CaD models can successfully generate the concept of remove given the context #2 . This result suggests that considering both local and global contexts are essential to generate correct descriptions. In our Wikipedia dataset, both the Local and LOG-CaD models can describe the word/phrase considering its local context. For example, both the Local and LOG-CaD models could generate american in the description for daniel o'neill given united states in context #1 , while they could generate british given belfast in context #2 . A similar trend can also be observed in Table 8, where LOG-CaD could generate the locational expressions such as philippines and british given the different contexts. On the other hand, the I-Attention model could not describe the two phrases, taking into account the local contexts. We will present an analysis of this phenomenon in the next section. In this section, we present analyses on how the local and global contexts contribute to the description generation task. First, we discuss how the local context helps the models to describe a phrase. Then, we analyze the impact of global context under the situation where local context is unreliable. Local context helps us (1) disambiguate polysemous words and (2) infer the meanings of unknown expressions. Can machines also utilize the local context? In this section, we discuss the two roles of local context in description generation. Considering that the pre-trained word embeddings are obtained from word-level co-occurrences in massive text, more information is mixed up into a single vector as the more senses the word has. While Gadetsky et al. (2018) de-1 2 3 4+ # senses 20 30 40 50 60 BLEU Global Local I-Attention LOG-CaD signed the I-Attention model to filter out unrelated meanings in the global context given local context, they did not discuss the impact of the number of senses has on the performance of definition generation. To understand the influence of the ambiguity of phrases to be defined on the generation performance, we did an analysis on our Wikipedia dataset. Figure 3(a) shows that the description generation task becomes harder as the phrases to be described become more ambiguous. In particular, when a phrase has an extremely large number of senses, (i.e., #senses  4 ), the Global model drops its performance significantly. This result indicates that the local context is necessary to disambiguate the meanings in global context. As shown in Table 2, a large proportion of the phrases in our Wikipedia dataset includes unknown words (i.e., only 26 . 79% of words in the phrases have their pre-trained embeddings). This fact indicates that the global context in this dataset is not fully reliable. Then our next question is, how does the lack of information from global context affect the performance of phrase description? Figure 3(b) shows the impact of unknown words in the phrases to be described on the performance. As we can see from the result, the advantage of LOG-CaD and Local models over Global and I-Attention models becomes larger as the unknown words increases. This result suggests that we need to fully utilize local contexts especially in practical applications where the phrases to be defined have many unknown words. Here, Figure 3(b) also shows a counterintuitive phenomenon that BLEU scores increase as the ratio of unknown words in a phrase increase. This is mainly because unknown phrases tend to be person names such as writers, actors, or movie directors. Since these entities have fewer ambiguities in categories, they can be described in extremely short sentences that are easy for all four models to decode (e.g., finnish writer or american television producer). As discussed earlier, local contexts are important to describe unknown expressions, but how about global contexts? Assuming a situation where we cannot obtain much information from local contexts ( e.g. , infer the meaning of boswellia from a short local context Here is a boswellia), global contexts should be essential to understand the meaning. To confirm this hypothesis, we analyzed the impact of the length of local contexts on BLEU scores. Figure 3(c) shows that when the length of local context is extremely short ( l  10 ), the LOG-CaD model becomes much stronger than the Local model. This result indicates that not only local context but also global context help models describe the meanings of phrases. In this study, we address a task of describing a given phrase with its context. In what follows, we explain existing tasks that are related to our work. Our task is closely related to word sense disambiguation ( WSD ) (Navigli, 2009), which iden-tifies a pre-defined sense for the target word with its context. Although we can use it to solve our task by retrieving the definition sentence for the sense identified by WSD , it requires a substantial amount of training data to handle a different set of meanings of each word, and cannot handle words (or senses) which are not registered in the dictionary. Although some studies have attempted to detect novel senses of words for given contexts (Erk, 2006; Lau et al., 2014), they do not provide definition sentences. Our task avoids these difficul-ties in WSD by directly generating descriptions for phrases or words. It also allows us to flexibly tailor a fine-grained definition for the specific context. Paraphrasing (Androutsopoulos and Malakasi-otis, 2010; Madnani and Dorr, 2010) (or text simplification (Siddharthan, 2014)) can be used to rephrase words with unknown senses. However, the target of paraphrase acquisition are words/phrases with no specified context. Although a few studies (Connor and Roth, 2007; Max, 2009; Max et al., 2012) consider sub-sentential (context-sensitive) paraphrases, they do not intend to obtain a definition-like description as a paraphrase of a word. Recently, Noraset et al. (2017) introduced a task of generating a definition sentence of a word from its pre-trained embedding. Since their task does not take local contexts of words as inputs, their method cannot generate an appropriate definition for a polysemous word for a specific context. To cope with this problem, Gadetsky et al. (2018) proposed a definition generation method that works with polysemous words in dictionaries. They presented a model that utilizes local context to filter out the unrelated meanings from a pre-trained word embedding in a specific context. While their method use local context for disambiguating the meanings that are mixed up in word embeddings, the information from local contexts cannot be utilized if the pre-trained embeddings are unavailable or unreliable. On the other hand, our method can fully utilize the local context through an attentional mechanism, even if the reliable word embeddings are unavailable. The most related work to this paper is Ni and Wang (2017). Focusing on non-standard English phrases, they proposed a model to generate the explanations solely from local context. They followed the strict assumption that the target phrase was newly emerged and there was only a single local context available, which made the task of generating an accurate and coherent definition difficult. Our proposed task and model are more general and practical than Ni and Wang (2017); where (1) we use Wikipedia, which includes expressions from various domains, and (2) our model takes advantage of global contexts if available. Our task of describing phrases with its context is a generalization of the three tasks (Noraset et al., 2017; Ni and Wang, 2017; Gadetsky et al., 2018), and the proposed method utilizes both local and global contexts of an expression in question. This paper sets up a task of generating a natural language description for an unknown phrase with a specific context, aiming to help us acquire unknown word senses when reading text. We approached this task by using a variant of encoder-decoder models that capture the given local context with the encoder and global contexts with the decoder initialized by the target phrase's embedding induced from massive text. We performed experiments on three existing datasets and one newly built from Wikipedia and Wikidata. The experimental results confirmed that the local and global contexts complement one another and are both essential; global contexts are crucial when local contexts are short and vague, while the local context is important when the target phrase is polysemous, rare, or unseen. As future work, we plan to modify our model to use multiple contexts in text to improve the quality of descriptions, considering the one sense per discourse hypothesis (Gale et al., 1992). We will release the newly built Wikipedia dataset and the experimental codes for the academic and industrial communities at https://github.com/shonosuke/ ishiwatari-naacl2019 to facilitate the reproducibility of our results and their use in various application contexts. The authors are grateful to Thanapon Noraset for sharing the details of his implementation of the previous work. We also thank the anonymous reviewers for their careful reading of our paper and insightful comments, and the members of Kitsuregawa-Toyoda-Nemoto-Yoshinaga-Goda laboratory in the University of Tokyo for proofreading the draft. This work was partially supported by Grant-in-Aid for JSPS Fellows (Grant Number 17J06394) and Commissioned Research (201) of the National Institute of Information and Communications Technology of Japan.
abstain result abstain abstain abstain objective abstain objective abstain objective method abstain abstain abstain result abstain abstain abstain abstain abstain abstain method objective result abstain method objective objective objective abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain other method method abstain abstain abstain method abstain abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain method method other abstain abstain method abstain abstain abstain abstain other abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain other abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain result abstain abstain abstain abstain result result abstain abstain abstain abstain abstain result method abstain abstain abstain abstain method method objective other method abstain other other other other other other other other method abstain other other abstain abstain objective method method abstain result other other other other