Unfamiliar terminology and complex language can present barriers to understanding science. Natural language processing stands to help address these issues by automatically defining unfamiliar terms. We introduce a new task and dataset for defining scientific terms and controlling the complexity of generated definitions as a way of adapting to a specific reader's background knowledge. We test four definition generation methods for this new task, finding that a sequence-to-sequence approach is most successful. We then explore the version of the task in which definitions are generated at a target complexity level. We introduce a novel reranking approach and find in human evaluations that it offers superior fluency while also controlling complexity, compared to several controllable generation baselines. Unfamiliar concepts and complex language can make understanding scientific information difficult for readers (Brossard and Shanahan, 2006; Shea, 2015; Martnez and Mammola, 2021), especially because understanding such terms is highly dependent on their domain knowledge. Given the wide variation in such knowledge, providing a one-size-fits-all definition may not be sufficiently understandable for all readers. We envision a software tool designed to aid readers with varying domain knowledge by automatically defining scientific terms. Such a tool would afford readers control over generated definitions, including their complexity. This hypothetical system motivates research on automated generation of scientific definitions and generation-time control of definition complexity. Prior work in generating definitions and personalizing generations to a reader falls short of these goals. Most definition generation has focused on common, general-usage words in English (Noraset A molecule that binds to a hydrophobic surface. Scientific terms rarely reach common usage (Shea, 2015; Britt et al., 2014) and the contexts in which their definitions might appear (e.g., a research paper) are often much more complex than general-purpose resources for definitions (e.g., dictionaries or standard word embeddings). Previous methods focused on reader personalization have aimed at generating based on a reader's prior knowledge and interests (Acharya et al., 2018; Murthy et al., 2021). These approaches work well when models can leverage a reader profile (Murthy et al., 2021) or incorporate reader feedback over time. However, in many cases a model might not have access to this additional information, such as for newcomers in an online forum discussing scientific findings (August et al., 2020a). We are interested instead in providing readers the ability to explicitly set definition complexity suited to their technical comfort (McNamara and Kintsch, 1996; Kintsch, 1994; Kim et al., 2016). We introduce a new task for generating definitions of scientific and medical terms with varying complexity (2; Joshi et al., 2017; Fan et al., 2019). Our dataset (3) is constructed from consumer medical questions and science glossaries containing words that vary in their complexity and frequency. We start by evaluating four modeling approaches for generating definitions, finding that, among them, a finetuned BART model is most successful at this new task (4). As a first step to adjusting definition complexity, we introduce methods to explicitly set definition complexity as either high or low at generation time. To our knowledge, this is the first paper using decoding-time controllable generation techniques on text complexity. We operationalize complexity based on readability and science communication research (Pitler and Nenkova, 2008; Gardner and Davies, 2013; Leroy et al., 2010) and evaluate several state-of-the-art controllable generation methods on this task (5). We also develop a new, lightweight method for controlling generation based on discriminator ranking. Our automatic and human evaluations show that our lightweight method is effective at varying complexity while maintaining high fluency and reducing factual errors. We release our dataset and code to encourage future work on this task. 1 2 Definition Tasks Generating definitions has been approached as a word-to-sequence task, where language models used a word's embedding to generate its definition (Noraset et al., 2017). Recent work used a sequence-to-sequence setup for generating definitions instead, where the defined word was a highlighted token in a sequence (Mickus et al., 2019). This conceptualization of definition modeling is an important starting point for addressing our task. However, new scientific terms are introduced regularly and many never appear in dictionaries or reach common usage (Shea, 2015; Britt et al., 2014), making it difficult to rely on general-purpose dictionaries (Kim et al., 2016). Scientific terms are also notoriously esoteric (e.g., hidden Markov model ) or else overload definitions of common words (e.g., transformer the model architecture versus transformer the electrical device), both of which complicate the use of standard word representations from pretrained models (Beltagy et al., 2019). ically, we frame our task as generating an answer to the question What is (are) X? This reframing allows us to leverage scientific definitions from more diverse sources (e.g., QA datasets) and to incorporate domain-specific knowledge into definition generation by including supporting information (3.2; Chen et al., 2017; Joshi et al., 2017). We collect a new dataset of definitions that are answers to the question What is (are) X? where X is a scientific term or concept (e.g., carbon nan-otubes ). These questions are drawn roughly equally from an existing QA dataset or templated from scientific glossaries. Medical consumer questions Ben Abacha and Demner-Fushman (2019) collected 47,457 medical questions from 12 National Institutes of Health (NIH) websites and collected them into the MedQuAD dataset. The dataset covers 37 different question types. Three question categories are focused on defining and providing information on medical terms: Information, How can I learn more, and Other information. Manual inspection of these question categories shows that all questions are of the form What is (are) X? or Do you have more information on X? Responses to the these questions begin with a brief definition of X. After filtering for this question type and removing questions with no answer due to copyright restrictions, we had 4,525 definitions. Wikipedia The MedQuAD questions are an excellent source of definitions, but only cover medical terms. Because we are interested in tackling scientific terms more broadly, we augment this set with terms drawn from Wikipedia science glossaries. 2 We extract all science-related terms and their definitions, yielding another 3,738 terms for a total dataset of 8,263 terms. We also explored using other QA datasets that included scientific information to expand our coverage of scientific domains outside of medicine, such as the Explain Like I am Five (Fan et al., 2019) and ARC science exam question 2 https://en.wikipedia.org/wiki/ Category:Glossaries_of_science 8299 Source Count Example Questions Example Definitions MedQuAD 4,525 What is (are) complement component 2 deficiency? datasets (Clark et al., 2018). We found these questions to be less focused on definitions, though future work might find ways to make use of them. We split our dataset into training, development, and test sets (8/1/1). Examples of terms in this dataset are in Table 1. 3.2 Support Documents We next collect scientific abstracts related to each term to allow models to incorporate related scientific knowledge (Fan et al., 2019; Clark et al., 2018). Specifically, given a term question (i.e., What is (are) X?), we query S2ORC (Lo et al., 2020), a corpus of over 81 million scientific articles, for the top 10 related abstracts. Query scoring and retrieval is done with Elasticsearch. 3 These abstracts are concatenated together and form the input along with the term question for our models (4). We use scientific abstracts, rather than general audience text like Wikipedia or the Common Crawl, for two reasons. First, scientific terms are originally introduced and most commonly used in research papers, making them the most reliable source for these terms. Second, terms can be contextual, having different meanings in common usage. Additional details for collecting the dataset and creating the support documents are in Appendices A.1 and A.2. Our goal is to create a definition dataset with (i) coverage of scientific and medical terminology and (ii) diverse levels of complexity, to support the application envisioned in 1. We conjecture that general-purpose dictionaries will lack coverage of such terms and tend to have complex definitions for those terms that they do include. Indeed, we found that less than 20% (191) of a sample of 3 https://www.elastic.co/ 1,000 terms in the medical consumer portion of our dataset have entries in the Merriam Webster Dictionary (MW). 4 The dictionary definitions also use substantially more academic vocabulary: an average of 39% (s.d. 12%) of words in those dictionary definitions were in the Academic Vocabulary List (Gardner and Davies, 2013)a list of words that occur more frequently in academic writing than common usagecompared to 29% (s.d. 12%) in the medical consumer definitions. Examples of definitions from our dataset and from MW are in Table 2. While complex definitions are not necessarily bad, we want diverse complexity levels in our input. While medical consumer questions tend to use fewer specialized terms than a dictionary, we also find that a random sample of 1,000 Wikipedia terms in our dataset use close to as much specialized terminology as a dictionary (37%, s.d. 12%). This provides us with a wider range of complexity levels than were we to use a single source of scientific definitions. We later explore how this exposure to different complexity levels in the input makes it possible to control the complexity of generated definitions (5.2). Our first goal is to generate fluent definitions that include relevant and accurate information about the term being defined. Because this is a new task and there are multiple reasonable approaches to generating fluent text (Prabhumoye et al., 2020), we experiment with four methods that have performed strongly in question answering and general-purpose definition generation and evaluate their effectiveness in this novel domain. For additional details on 4 For this analysis, we exclude the Wikipedia science glossary terms since Wikipedia is also often used as a generalpurpose resource of definitions, and the Merriam Webster API restricts us to 1,000 queries. Sequence-to-Sequence: BART ( BART SD and BART NO SD ) BART (Lewis et al., 2020) has been used to define general English terms in context (Bevilacqua et al., 2020) and reached state-of-the-art results on the Explain Like I am Five (ELI5; Lewis et al., 2020) QA dataset, which includes some questions requiring scientific knowledge (e.g., What is a Turing Machine and why is it so important?). We experiment with finetuning the BART pretrained model on our task and dataset in two ways. In the first, BART is trained with the term question concatenated with the supporting document (referred to as BART SD ). In the second, we train a BART model with just the term question and definition answer, without the support documents ( BART NO SD ). This second version is included to assess how important the support documents are for generating definitions of scientific terms. We use BART-large as our base model for both versions. 5 Out-of-the-Box Causal Language Modeling ( OOTB GPT -2 and OOTB GPT -3) Recent work has also shown that large pretrained causal language models, such as GPT-2 and GPT-3, can generate fluent answers to factual questions without finetuning (Brown et al., 2020). We experiment with using both GPT-2 and GPT-3 out-of-the-box ( OOTB GPT -2 and OOTB GPT -3). We use GPT-2 medium 6 and GPT-3 davinci 7 for 5 https://huggingface.co/facebook/ bart-large 6 https://huggingface.co/gpt2-medium . We obtain similar results when using GPT2-large. 7 https://beta.openai.com/ these experiments. For OOTB GPT -3, we evaluate with 100 terms due to OpenAI API limits. For generation, we follow the few-shot setting proposed in Brown et al. (2020) and prepend two held-out question term and definition pairs before each generation. We do not include the support documents in this few shot setting since doing so extends beyond GPT-2's context window of 1024 tokens and preliminary results showed that the additional text led to fewer definitions and more repetition from the abstracts. Finetuning GPT-2 ( FT GPT -2): Because OOTB GPT -2 and OOTB GPT -3 involve no finetuning or use of the support documents, we suspect that they will underperform BART SD . We experiment with finetuning the GPT-2 medium model ( FT GPT -2) with the question and support document, separated by new special tokens. Information Retrieval ( OOTB BIDAF ): Information retrieval (IR) methods are an important part of many open-domain QA systems and have presented a strong baseline in scientific question answering (Clark et al., 2018). We experiment using a pretrained BiDAF model (Seo et al., 2018) to extract the highest scoring span in the support document based on the term question ( OOTB BIDAF ). We use AllenNLP's implementation of BiDAF trained on SQuAD. 8 4.2 Results Table 4 shows the ROUGE scores and BERTscore for each modeling method on the development set of our dataset. models. OOTB GPT -3 performs surprisingly well, outperforming even FT GPT -2. BART NO SD also performs closely to BART SD , suggesting that while the support documents can boost performance in this task, the effect can be small. OOTB BIDAF extracts spans that don't answer the question. Table 3 provides examples of the generated definitions for each modeling approach. BART SD provides the most concise answer while also remaining informative, compared to FT GPT -2's definition, which is circular (e.g., Acanthoma (cancer) is a type of cancer). While most models show impressive background knowledge, there is evidence of incorrect or hallucinated information, such as acanthoma being a type of skin cancer ( OOTB GPT -2 and BART NO SD ), these hallucinations are marked in Table 3. We explore the amount of hallucinated information further in 7.2. For the rest of the paper we use the BART SD model since it outperforms other methods. Automatically generating definitions is an important first step in supporting readers who come across unfamiliar scientific terms, but individuals can have different tolerances for the complexity of a definition depending on their domain knowledge (Britt et al., 2014). The models we tested in 4 were not trained to vary the complexity of definitions; they do not adapt definitions to different readers. Here we explore how to control the complexity of generated definitions. Controlling or guiding text generation is an active research area with important applications like toxicity control (Gehman et al., 2020) and language debiasing (Ma et al., 2020). For a review, see Prab-humoye et al., 2020. To the best of our knowledge, ours is the first work to evaluate decoding-time controllable generation methods for text complexity. One task that has considered changing text complexity is text simplification. Work on text simplification has mostly used a machine translation setup based on parallel corpora (Zhu et al., 2010; Cao et al., 2020) to translate complex sentences into simple ones. These parallel corpora are rare and often expensive to create (Xu et al., 2015). This setup also assumes an input text to be simplified (Surya et al., 2019), whereas our task expects that the text will be generated with varying complexity. Below we describe prior methods, used as baseline generation control methods. In each case, we focus on a binary distinction between low complexity and high complexity definitions, leaving more fine-grained distinctions to future work. We also introduce a novel lightweight approach based on reranking candidate generations in 5.2. Additional details for training are in Appendix A.4. Plug-and-play language models PPLM (Dathathri et al., 2020) is a technique to guide generation using the gradients of a classifier for a 8302 particular desired text attribute. At each generation step, the classifier's gradients are used to update the language model's hidden representations. Due to the computational expense of PPLM, we evaluate with 100 randomly sampled test set terms. We train our attribute classifier on sentences from scientific journal abstracts and scientific news articles. Journal abstracts are sampled from the ArXiv dataset (Clement et al., 2019) and used to guide to more complex language. Scientific news articles are sampled from a corpus of science news articles (August et al., 2020b) and used to guide towards less complex language. Generative discriminators The GeDi method (Krause et al., 2021) uses a class-conditioned language model trained on text with a certain desired (or undesired) feature (e.g., toxicity) to guide generation. At each generation step, the model provides next token probabilities to the generator via Bayes' rule. We train a new GeDi on the same dataset of science news and journal articles as for PPLM. Ensemble of language models DExperts (Liu et al., 2021) combines multiple pretrained language models in an ensemble of experts and anti-experts. Specifically, a base language model is combined with a language model trained on text with desirable attributes (expert) and text with undesirable attributes (anti-expert). At generation time, the base model's logits are combined with the difference of the expert's and anti-expert's logits. Our expert and anti-expert are pretrained BART-large models that we continue to pretrain on the data used to train the PPLM discriminator. One model is pretrained on the journal abstracts and one on the science news articles. To generate more complex definitions, the expert is the model trained on journal abstracts while the anti-expert is the model trained on science news. To generate less complex definitions, the roles are reversed. We introduce a new, lightweight method to generate definitions with different complexity via reranking. Past work has explored selecting candidate generations based discriminator scores to control for specific topics or discourse structure but found that it did not provide strong control (Dathathri et al., 2020; Gabriel et al., 2021). Because our generation task does not require topic shifts and our input has naturally varying complexity (3.3), we adapt this method by scoring and selecting candidates based on complexity discriminators. Specifically, at test time we use our BART model ( BART SD ) to generate 100 candidate definitions for each definition. We then rerank these candidate generations based on logits from a discriminator trained to distinguish scientific journal text from science news text. While this method requires regenerating the definition many times, it does not require gradient or probability distribution updating during generation or any prior pretraining, allowing for much greater flexibility during generation (e.g., generating from a language model without access to vocabulary logits during generation). We consider two discriminators. Both are trained on the same dataset of science news and journal articles as PPLM. BERT We use the SciBERT uncased pretrained model (Beltagy et al., 2019). For more complex definitions we select definitions with high predicted probability for journal text, and for less complex definitions we select definitions with high prediction probability for science news text. Linear We also experiment with using a linear SVM classifier. The SVM's features are complexity measures drawn from science communication and readability literature, discussed in 5.3. The complexity of scientific writing is affected by many factors and it is difficult to operationalize it into a single dimension. We therefore use multiple measures of scientific writing complexity based on prior work in science communication and readability. These measures are not meant to be an exhaustive list (for a review, see Pitler and Nenkova, 2008), but a selection of measures that capture different elements of complexity important to definitions. Table 17 in the Appendix has examples of model outputs that scored either very high or very low for each measure. Scientific language is often associated with language formality (Lahiri, 2016; Heylighen et al., 1999). This might lead to some methods introduced in 5.1 to generate definition with higher formality when guided towards higher complexity. We therefore focus our measures on aspects of complexity important for reader comprehension (e.g., unfamiliar terminology or dense text) rather than the formality of the definition. ways. Five of them are the features in our linear SVM reranker. We also use them as a preliminary automatic evaluation of the various controllable generation approaches in 5.1 and 5.2. Obviously, we expect the linear SVM reranker to outperform the other approaches in this automatic evaluation since it was trained with these complexity features; it should be considered something like an upper bound for these complexity measures. Our human evaluations (6.2 and 7) provide a more complete picture of the systems' performance. Academic Vocabulary List (AVL) occurrences The AVL is a list of academic vocabulary drawn from corpora spanning many scientific disciplines (Gardner and Davies, 2013). We measure the fraction of AVL words in a generated definition. Thing Explainer out-of-vocabulary The popular book Thing Explainer explains scientific concepts using only the 1,000 most frequent words in English (measured by Wiktionary's contemporary fiction frequency list; Munroe, 2017). 10 We measure the fraction of words in the definition outside of the top 1,000 used in Thing Explainer . Function words In health communication, function words (e.g., prepositions, auxiliary verbs, or question words) positively correlate with perceived and actual readability (Leroy et al., 2008, 2010). Sentence length Sentence length is a commonly used metric for document level complexity and is part of many classic readability measures (Pitler and Nenkova, 2008; Feng et al., 2010). While we set a maximum generation length for our definitions (64 tokens), we enable early stopping. While longer sentences are often considered more complex, we hypothesize that in our dataset longer definitions will be associated with less complex 10 https://en.wiktionary.org/wiki/ Wiktionary:Frequency_lists/Contemporary_fiction language due to elaborative simplification, where complex terms are explained as a way of simplifying them (Srikanth and Li, 2021). Language model perplexity Language model perplexity has been found to correlate with perceived and actual reading difficulty (Pitler and Nenkova, 2008; Collins-Thompson, 2014). We use the GPT model to measure language model perplexity, as it was trained on common English (as opposed to scientific text). Flesch-Kincaid grade level This score (FK) uses simple calculations based on sentence length, word length, and syllable counts (Kincaid et al., 1975). Although findings are mixed on how well the FK predicts readability in science or medical documents (Leroy et al., 2008), it is a standard, widely used measure of text complexity (Redmiles et al., 2019). The FK expects a document with multiple sentences, but our definitions are a single sentence. To address this, we calculate the FK based on the concatenation of all definitions generated by a particular method. For the same reason, we do not include the FK score as a feature in our SVM reranker (5.1). As a preliminary analysis of complexity using our measures, we evaluate how the journal abstracts and science news articles used for guiding complexity generation (described in 5.1) vary across our measures. Table 5 includes a row representing the difference in each measure between the training set of journal abstracts and science news articles (Jour-nal News). We see that all the measure behave in the expected direction except sentence length, which goes in the opposite direction. This might signify that journal abstracts still use longer sentences even if science news articles are explaining complex topics to simplify them (Srikanth and Li, 2021). Here we evaluate how well our baseline and novel generation control methods can vary the complexity of definitions. For each generation method, we generate and evaluate 10 definitions for each term. We automatically evaluate each control method by calculating the difference in each complexity measure (5.3) for the high and low complexity generations. Table 5 details these differences. While each measure captures a different element of complexity, counting the number of words outside of the top 1,000 most common English words (TE) seems to be one of the most consistent measures, with all higher complexity generations having differences in the expected direction. DExperts and the BERT reranker have the largest differences, with 5% and 4% more words per sentence. Higher complexity generations also have higher GPT perplexity, with DExperts having the largest difference. The two rerankers (BERT and SVM) perform better than other models on most measures. This is unsurprising for the SVM since it was trained with these complexity features, but it is interesting that reranking with the BERT classifier also provides effective control over complexity. Table 6 provides example generations based on each approach. Automatic classification of text complexity is difficult and domain-specific (Collins-Thompson, 2014; Redmiles et al., 2019); even in combination, we believe the measures in 5.3 are insufficient for a full evaluation of our approaches. We therefore carry out a human evaluation to assess how each method influences perceived definition complexity. We select the models that performed best overall in our automatic evaluation: DExperts, GeDi, and the SVM reranker. 11 We randomly sample 50 terms from our test split to evaluate. We use a high and low complexity generation from each model, leaving us with 50  2  3 = 300 definitions. We broke down complexity into two ratings: how complicated a definition was and how difficult to understand the definition was. For each, participants rated definitions on a 14 Likert scale. We recruited participants on Amazon Mechanical Turk. Each participant was payed US$0.50 cents based on US$10 dollars/hour. This study was approved by our institution's internal review board. Participants 233 participants took part in our evaluation (mean age 35 years, s.d. 11). Table 18 in the Appendix provides more details on their demographics. We removed 4 participants due to low effort responses (i.e., responding to all prompts with the same rating within 15 seconds). Results Figure 2 shows the average ratings for each model type. DExperts generations differentiate most between high and low complexity. GeDi definitions behave in a way that is the opposite of what we expected, with the low complexity generations rated as more complicated and difficult to understand than the high complexity generations. The SVM-reranked definitions perform in the expected direction, with high complexity generations 11 We do not include PPLM in this analysis due to its computational cost and similar performance to GeDi. being rated as more complicated and difficult to understand. Examples of ratings and raw counts are in Table 19 and Figure 4 in the Appendix. Our results suggest that our reranking method is a simple intervention that can control complexity with similar performance as other state-of-the-art methods. However, definitions of scientific terms also must be fluent, relevant, and factual. Factuality can be especially difficult to achieve in generations (Maynez et al., 2020). In science communication such failures could spread misinformation with fluent but incorrect definitions (Britt et al., 2019). We do two additional human evaluations for fluency and relevance (7.1), and factuality (7.2). We used two trained annotators, one of them an author, to rate the same 300 definitions used in the complexity evaluation (6.2). Neither annotator saw the model generations before evaluation or knew which method had generated each definition. Annotators rated definitions for fluency and relevance using 14 Likert scales (1 = Not at all to 4 = Very). Table 7 shows the average fluency and relevance ratings. The SVM-reranked definitions were rated close to Very fluent and relevant (both above 3.5 on a 4 point scale), and significantly more fluent compared to GeDi ( t 198 = 5 . 99 p < 0 . 001 , Cohen's d = 0 . 60 ) and DExperts ( t 198 = 18 . 85 p < 0 . 001 , d = 1 . 88 ). For each definition, annotators identified if there was any factually incorrect information in the definition (a binary label) and if so, rated how extensive these errors were on the same 14 scale. Table 7 reports on the average rating for how extensive these errors were. Below we report on the binary label. Overall 60% of our generations were labeled as factually incorrect by at least one annotator (40% by both). The SVM had significantly fewer factual errors (38% by one annotator, 16% by both), compared to GeDi (52% and 33%, t 198 = 4 . 71 p < 0 . 001 , Cohen's d = 0 . 47 ) and DExperts (86% and 67%, t 198 = 12 . 29 p < 0 . 001 , d = 1 . 24 ). We introduce a new task and dataset for generating definitions of scientific terms with controllable complexity as a way of adapting to different readers' scientific background. We evaluate conventional generation methods and introduce a lightweight approach of reranking candidate generations based on a discriminator to control complexity. We find that this reranking is effective at controlling text complexity while also maintaining fluency and factuality. We release our dataset and code to encourage more work on making scientific terms more accessible to readers of diverse background knowledge. 12 9 Ethical Considerations The goal of this paper is to enable a wider audience of readers to understand and engage with scientific writing. A risk, though, is that such attempts might instead widen the gap to accessing scientific information. The texts in the datasets we train our models on are in General or Academic American 12 https://github.com/talaugust/ definition-complexity 8306 English. Many people, especially those who have been historically underrepresented in STEM disciplines and medicine, may not be comfortable with this dialect of English. This risks further alienating the readers we hope to serve. This is a common issue in NLP systems (Sap et al., 2019), since the majority of datasets are in General American English. An important and exciting direction in NLP is making models more flexible to dialects and low-resource languages (e.g., the ACL 2022 special theme being Language Diversity). While our results suggest that the lighter control of reranking generations leads to less hallucinated information, strong supervision of definition factuality is important for any future deployment of such a system. While hallucinated information can be damaging in any generation context, incorrect scientific definitions could mislead readers and potentially contribute to broader scientific misinformation. Furthermore, a bad actor could use these models to generate fluent but incorrect definitions at scale, potentially contributing to misinformation campaigns with a veneer of scientific language (Britt et al., 2019). We trained our models on data we believe is trustworthy (e.g., questions and answers from NIH websites); and we release our training data and models to allow for further work on encouraging factuality in these model generations. We thank Anita Silva for her help annotating definitions and the Mechanical Turk annotators for their work on the project. We also thank the anonymous reviewers and members of the UWNLP and DUB community for their helpful feedback. This work was supported in part by the Office of Naval Research under MURI grant N00014-18-1-2670 and by a Twitch Research Fellowship.
abstain abstain objective objective objective objective abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method objective objective objective method objective result method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method other abstain abstain method method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain method abstain abstain abstain method other method abstain method abstain method abstain method other method other method method method method method abstain method method method abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain method method other abstain abstain abstain method abstain abstain abstain method abstain abstain abstain method abstain method method method abstain method abstain method method method abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain method abstain method abstain method abstain abstain abstain other abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method result method objective abstain other abstain abstain abstain abstain result abstain abstain method other other other