Detecting biased language is useful for a variety of applications, such as identifying hyperpartisan news sources or flagging one-sided rhetoric. In this work we introduce WikiEvolve, a dataset for document-level promotional tone detection in English. Unlike previously proposed datasets, it contains seven versions of the same article from Wikipedia, from different points in its revision history; one with promotional tone, and six without it. We adapt the gradient reversal layer framework to encode two article versions simultaneously, and thus leverage the training signal present in the multiple versions. In our experiments, our proposed adaptation of gradient reversal improves the accuracy of four different architectures on both in-domain and out-of-domain evaluation. Maintaining a neutral point of view is a desideratum in many communication channels, e.g. news articles, scientific writing, and encyclopaedias. Biased writing detection can help reduce the distribution of content which contains unfair representations of a topic. For this reason, datasets and methods have been developed to automate it. A number of studies have approached biased writing detection in the context of news media (e.g. Fan et al. (2019); Chen et al. (2020); Frber et al. (2020)), primarily considering political stance and partisanship. However, biased writing also arises in other settings. In Wikipedia, the online encyclopaedia, it manifests itself in the form of promotional tone which violates the cornerstone neutral point of view policy of the platform. The latter allows users to flag articles with such policy violations by adding tags to the article mark-up, which are retained in its edit history. Leveraging  This work was initiated during an internship at the Wikimedia Foundation. this process, Recasens et al. (2013) and Aleksandrova et al. (2019) have released datasets of words and sentences which were altered in subsequent revisions, thus facilitating model development for word/sentence level bias detection. In this work, we propose an alternative data collection methodology for document-level promotional tone detection, We sample multiple versions of the same article in Wikipedia and present WikiEvolve 1 , a dataset of 68,498 labelled articles for this task. These articles are arranged into 13,887 sample sets, where each set contains multiple versions of the same article: one version tagged as having a promotional tone problem, and up to three versions respectively from before the tag was added and after it was removed. This is illustrated in Fig. 1; the second version was labelled as containing 1 github.com/christinedekock11/wiki-ev olve 5601 promotional tone (positive), whereas the first and third versions were considered negative. In contrast with Recasens et al. (2013) and Aleksandrova et al. (2019), we choose to perform classification at the level of documents rather than sentences or words. Our motivation is that classifying a sentence out of context as biased is known to be difficult and prone to subjective judgements, while higher inter-annotator agreement is achieved at the document-level (Chen et al., 2020). Recasens et al. (2013) similarly found that identifying promotional tone at the word-level is challenging, with Mechanical Turk workers achieving 37% accuracy on this task. We hypothesise that there are article-level features which provide corroborating evidence to the intentions of the writer, which isolated sentences might not capture. We also see evidence of this in our own data  for instance, in Fig. 1, the mention of leading specialist in version 3 is dubious but justifiable; however, in version 2, it contributes to an overall assessment of biased writing. To make better use of the training signal available in the multiple versions per article in WikiEvolve, we adapt gradient reversal (Ganin and Lempitsky, 2015). The latter entails adding an auxiliary task during training which shares the input encoder with the main task and is optimised concurrently, but its gradients are reversed during back-propagation. The model is therefore discouraged from learning features which are useful for the auxiliary task and assumed harmful for the main task. Our adaptation operates on pairs of samples rather than individual texts, and we define the auxiliary task as classifying whether two samples originated from the same article. The features we learn are therefore more likely to be informative of the tone, but not of the content. In our experiments, gradient reversal improves the accuracy of all four architectures of increasing complexity. On a bag-of-words encoding followed by two neural network layers, the PR-AUC score improves from 0.60 to 0.64. Using a hierarchical attention network, performance is increased from 0.63 to 0.65. This illustrates that the additional structural information WikiEvolve provides can be utilised to improve performance on this task. To further assess the ability of gradient reversal to improve performance by encouraging models to learn features that do not rely on the topic or content, we also tested our models on out-of-domain data from the SemEval 2019 Shared Task on Hyperpartisan News Detection (Kiesel et al., 2019). Our results show that GRL training improves our accuracy on this dataset from 0.714 to 0.785. A number of studies have utilised Wikipedia to develop labelled datasets for content-related issues, including promotional tone detection. Wikipedia has several favourable characteristics which enable this form of data collection. Firstly, articles evolve over time through different versions. Secondly, the chronological revision history of each article is preserved and open-sourced 2 , meaning that the evolution of an article can be retrieved. Finally, the platform's decentralised quality control system allows users to tag articles that violate the platform's content policies, to warn readers of such issues and to attract the attention of editors to fix them. These tags are removed from the article once the problem is resolved, but they are preserved in the article's edit history. A more details on Wikipedia's policy violation tags see Anderka et al. (2012). In this context, a revision of an article which contains a tag is considered a positive instance of that specific policy violation. Different methods have been proposed for sampling negatives. A popular approach is to find revisions of the same or other articles which do not contain the tag. However, this approach can introduce noise, as the absence of a policy violation tag from an article does not guarantee that the problem is not present. This characteristic of template-based Wikipedia datasets has been noted in previous work, e.g. Anderka et al. (2012); Bhosale et al. (2013); Orizu and He (2018). Another option is to look to other articles which are known to represent well-written content. For instance, Anderka et al. (2012) and Bhosale et al. (2013) select negatives from Wikipedia's list of featured and good articles. However, these articles are of a higher quality generally and therefore have other distinguishing characteristics, which may be misleading if the goal is to detect policy-violating content. Additionally, sampling negatives from different articles may introduce a topical bias. Our data extraction methodology consists of (i) finding articles tagged by a Wikipedia editor as having a promotional tone problem at some point 2 A Creative Commons Attribution-Share-Alike License 3.0 applies. in their edit history, (ii) selecting the revision where such a tag was added as a positive sample, and (iii) sampling negatives from revisions which did not contain the template. Finding promotional tone tags To identify tags of interest, we refer to the Wikipedia category ar-ticles with a promotional tone (Wikipedia, 2021a) and identify the quality tags which most frequently occur in this articles of this category. These are ad-vert, autobiography, fanpov, peacock and weasel. Each of these tags describes a different type of promotional tone issue, for which the definitions are contained in Appendix A. We then use regular expressions to collect all revisions which contain variations of these tags in the WikiText data lake (Wikipedia, 2021c). Finding tag addition events Once incidences of promotional tone tags have been identified, we use the WikiHistory data lake (Wikipedia, 2021b) to find the full edit histories of these articles. For each article, we then identify the point in its edit history where a tag was added, and consider this version of the article as the positive sample. We exclude cases where the tag addition edit was reverted 3 by another editor. The article text at this timestamp is retrieved from the WikiText data lake. Sampling negatives For each positive sample, we select negatives from the revision history of the same article. We consider as candidates all revisions which were not reverted, and which took place within 30 revisions (chronologically sorted) of the tag addition event. This is intended to ensure that the negative samples are of the same approximate stage of article development as the positive sample. We exclude the revision immediately before the tag addition event, as it is this version which prompted the tag to be added. Up to three revisions (depending on availability) are selected at random from these candidates, before and after the positive. We refer to such a set of samples as a sample set . The negatives sampled before the tag addition are denoted neg_pre , and those from after are denoted neg_post . The number of samples per tag and class are shown in Table 1. We split the data into train, test and validation sets with a ratio of 70-20-10. The datasets are stratified to contain 3 From the platform guidelines: On Wikipedia, reverting means undoing or otherwise negating the effects of one or more edits, which results in the page (or a part of it) being restored to a previous version. samples from each tag type (set out in Table 1), and samples from the same sample set (i.e. revisions of the same article) are kept in the same split. Although this work only considers promotional tone detection in English, the data collection methodology and training framework we propose could be extended to other languages on Wikipedia, as is done in Aleksandrova et al. (2019). As discussed in Sec. 2, Wikipedia tag-based datasets are known to contain a certain level of noise. To counteract this, we have implemented three measures: ensuring that the negatives are from the same stage of article development, sampling from different points in the same article's edit history, and sampling negatives before and after the positive. However, there is still a risk of including false negatives, i.e. articles not tagged as containing promotional tone even though they do. An example of such a case from our dataset is shown in Fig. 2. Despite containing non-neutral phrases such as hit show, made quite an impression, and prove[d] herself to be intelligent, the neg_pre (first) sample is not tagged as containing biased language. It does however contain some information that re-flects negatively on the subject (for the wrong reasons). This is removed in the positive (middle) sample, and more overtly biased descriptions are added (quick wit, educational background, amazing looks, bubbly personality and easy on the eye appearrance). In the neg_post negative sample, the problematic phrasing is removed. We perform manual validation of our dataset to estimate how frequently false negatives are included. We perform two types of validation: pairwise and independent prediction. For the former, the task is to rank two samples (i.e. revisions of the same article) as to which is more promotional. 40 articles, consisting of 20 positive-negative pairs in random order, are evaluated by two of the authors. The orderings of the annotators agreed with the assigned labels for respectively 16/20 and 14/20 pairs, with a Cohen's Kappa score of 0.79 indicating substantial agreement. This suggests that the collected data contains a trustworthy signal for comparing the extent to which two texts are promotional. Since the task we are mainly interested in is text classification, rather than ranking, we also perform an evaluation on individual samples, annotating 30 samples of each type (positive, neg_pre and neg_post ). The concurrence with the mined labels of the neg_pre and neg_post annotations are shown in Table 2. This task appears to be more challenging compared to the pairwise comparison, with both annotators achieving lower scores and a lower inter-annotator agreement Kappa score of 0.4805, indicating moderate agreement. A reason for this may be the subjective nature of the task, as illustrated by Chen et al. (2020). Our evaluation indicates that the negative samples from before the tag was added contain more noise, compared to those sampled after it was removed. This can be attributed to the active removal of the tag by an editor in the version after Annotator neg_pre neg_post A1 1230 1430 A2 1430 2230 Total 2460 3660 Table 2: Agreement of two authors with mined labels of negative sample annotations. the tag was added ( neg_post ), which indicates that the problem is resolved, while the lack of a policy violation tag in earlier versions ( neg_pre ) does not guarantee lack of promotional tone. However, ignoring the neg_pre samples altogether would expose the temporal bias mentioned in Sec. 1: if negatives are always sampled chronologically after positives and from a more developed version of the article, spurious correlations may be inferred. Based on these insights, we have chosen to include the automatically mined neg_pre samples in training, but to create a separate set of manually validated neg_pre samples for evaluation. Thus, we randomly selected 100 neg_pre samples from the original test set and verified whether they represent a neutral writing style. 42 of the 100 samples were confirmed as true negatives. We balance these negatives with their corresponding positive samples, and refer to this dataset as ValidNegPre . Gradient reversal training (Ganin and Lempitsky, 2015) jointly optimises two classifiers which rely on a shared underlying encoder model: (i) a label predictor for the main task, which predicts class labels and is used during both training and test time, and (ii) a domain classifier, which predicts either the source or the target domain during training as the auxiliary task. The parameters of the encoder model are optimised to minimise the loss of the main task classifier while maximising the loss of the domain classifier. This is achieved through a gradient reversal layer, which leaves the input unchanged during forward propagation and reverses the gradient by multiplying it by a negative scalar during the backpropagation. This approach is motivated by theory on domain adaptation, which suggests that a good representation for cross-domain transfer is one for which an algorithm cannot learn to identify the domain of origin of the input observation (Ben-David et al., 2010). Our adaptation of this framework, shown in Fig. 3, differs from Ganin and Lempitsky (2015) in that it considers two text inputs concurrently ( x and x (cid:48) ), as opposed to one. f represents a neural network encoder with parameters  1 . f encodes the two texts independently, to produce z and z (cid:48) : z = f ( x ;  1 ); z (cid:48) = f ( x (cid:48) ;  1 ) (1) The network then splits into two branches. The primary (bottom) branch consists of a neural network model g , with parameters  3 , which produces promotional tone predictions  y T 1 and  y T 2 for the two samples:  y T 1 = g ( z ;  3 );  y T 2 = g ( z (cid:48) ;  3 ) (2) The auxiliary branch concatenates the two input encodings as [ z, z (cid:48) ] , and then the similarity classifier h , a neural network parameterised by  2 , provides a prediction  y sim of whether the two samples originate from the same Wikipedia article:  y sim = h ([ z, z (cid:48) ];  2 ) (3) Our intention with this task is to encourage the encoder f to learn features that are topic agnostic. This should allow for better generalisation across datasets, as well as to avoid learning spurious correlations due to topical biases in the data. The encoder and classifier models are trained simultaneously. Given a set of training samples D = [ x 1 , ..., x N , y 1 , ..., y N ], we construct M pairs with indices P = ( i, j ) : i, j  [1 , ..., N ] . The process for generating these pairs is described in Sec. 6. Then, the loss is given by: L (  1 ,  2 ,  3 , D, P ) = 1 MM (cid:88) m =1 (cid:18) LT ( x P 1 , x P 2 ;  1 ,  2 )  L Sim ( x P 1 , x P 2 ;  1 ,  3 ) (cid:19) , (4) such that the loss with respect to the similarity label is maximised, while the loss with respect to the promotional tone label is minimised. is a scalar which controls the weight of the loss from the adversarial task, and LT = LT 1 + LT 2 . During testing, only the feature encoder and the main task branch are retained to perform the tone classification task:  y = g ( f ( x ;  1 ) ,  3 ) . Recall that the model encodes and predicts on the two input samples independently during training. We can therefore obtain predictions for individual test samples (rather than pairs), as is the more general case in other models and datasets for this task. The two classifier models, g and h , are both MLP models. The feature encoder (indicated as f in Fig. 3) is responsible for producing an embedding of an article to be used in both the main and auxiliary task. We evaluate four options:  Bag-of-words ( BoW + MLP ): a bag-of-words representation of an article is propagated through a multilayer perceptron (MLP) to obtain an embedding,  Averaged embeddings ( AvgEmb + MLP ): GloVe embeddings (Pennington et al., 2014) for every word in the article are averaged, followed by an MLP model,  Hierarchical Attention Network (HAN) (Yang et al., 2016): word embeddings are processed using an LSTM layer followed by an attention mechanism to build up sentence embeddings. Sentence embeddings are similarly combined to form an article embedding. Longformer (Beltagy et al., 2020): A transformer-based model, adapted for long-form documents. We finetune the pretrained longformer-base-4096 model. For the GRL models, we further experiment with the weights of the main versus auxiliary task  on the validation set, finding that weighting the outputs equally yields the best results. We compare the GRL training approach with the standard method of training the classifier with each feature extractor model. This is equivalent to training with the inference model in Equation 5; the auxiliary branch is removed and one sample is processed at a time. Implementation details are provided in App. B. 6.2 Metrics For each model, we report two metrics:  PR-AUC : The area under the precision-recall curve, which provides an aggregate measure of performance across all possible classification thresholds (Davis and Goadrich, 2006). Perfect performance is 1, and a random classifier would receive 0. Accuracy : The percentage of samples which are correctly classified, using a classification threshold based on Youden's J statistic (Fluss et al., 2005), which maximises the true positive rate and minimises the false negative rate. In order to train the main task we require samples with and without a promotional tone (i.e. positive and negative labels). To train the auxiliary we require both matched pairs (originating from the same sample set / article) and unmatched pairs (originating from different sample sets). Therefore, we include a number of different pairing configura-tions. Firstly, given a training set consisting of K sample sets, we collect K positive-negative matched pairs. This means that we need to select one negative sample and one positive sample from each sample set. There are multiple negatives in every sample set, so we sample at random from all neg_pre and neg_post samples. There is only one positive per sample set, so this sample is used. We also collect K positive-negative unmatched pairs. We further include K matched and K unmatched negative-negative pairs. Finally, we include K positive-positive unmatched pairs. It is not possible to add positive-positive matched pairs, as there is only one positive per sample set. Using this pair selection method, there are 7 K samples for the tone classification task and 3 . 5 K pairs for the similarity classification task, for a total of 48762 articles. For the baseline models, without GRL, only one sample is used at a time during training; thus, we retain the data generation method described above (to ensure the results are comparable), but ignore the pairings. The training dataset is slightly unbalanced, with a ratio of 4:3 of negatives to positives for the similarity classification task and a ratio of 4:3 of unmatched to matched pairs. The numbers of samples per label and their origin are shown in Table 7 in App. C. Our validation and test sets consist of only positive-negative matched pairs, one from each sample set, and thus are fully balanced. As motivated in Sec. 4, for the main test set (denoted FullTest ) negatives are only selected from the neg_post samples. For the ValidNegPre test set, all negatives are manually validated. The text preprocessing steps are described in App. B. 7 Results The results from our evaluation on the FullTest are in Table 3. We observe that models trained with GRL consistently outperform models trained without it, on both the accuracy and PR-AUC metrics. All improvements, except for the Longformer, are 5606 Model PR-AUC Accuracy BoW + MLP 0.6019 0.5913 BoW + GRL 0.6409 0.6102 AvgEmb + MLP 0.6129 0.5848 AvgEmb + GRL 0.6415 0.6084 HAN 0.6271 0.5968 HAN + GRL 0.6459 0.6102 Longformer 0.6798 0.6392 Longformer + GRL 0.6984 0.6432 Table 3: Results using GRL training on FullTest . statistically significant at the  = 0 . 05 level, using the permutation test to compare PR-AUC values. Larger gains are observed for the BoW+MLP and AvgEmb+MLP models, compared to the HAN and Longformer models. A possible explanation for this is that the simpler models rely only on word-level information, and thus more susceptible to topical biases which GRL mitigates. These results support the motivation behind our data collection method and training framework: by incorporating our knowledge of how samples are related in our dataset and training, models are exposed to different versions of the same content (with and without promotional tone), and can therefore better learn features that are more effective for detecting promotional tone, compared to models that ignore this information. Given our discussion in Sec. 4, we also evaluate the GRL approach on a separate, validated test set, which uses neg_pre rather than neg_post negatives (denoted ValidNegPre ). In this evaluation, we are particularly interested in the effect of excluding neg_pre samples during training. The total number of samples in the training set remains the same as for experiments already reported, but all negatives are sampled from the neg_post samples for each sample set during training. We compare our best model (Longformer+GRL) under these conditions. For brevity, in Table 4 we only show the PR-AUC values, but the same trends hold for accuracy. The original configuration is shown in the top left of the table; with training data including both neg_post and neg_pre , and testing on F ullT est . We note that, using the same training data, the PR-AUC score is slightly lower on the ValidNegPre set (top right) compared to the FullTest set, indicating that these samples may be more difficult to classify correctly. The effect of excluding neg_pre samples during training is shown in the second row. The two training settings achieve similar performance on the FullTest test set, however, the performance on the ValidNegPre dataset is markedly lower when excluding neg_pre samples. This supports our motivation for including neg_pre samples during training, as described in Sec. 4, i.e. that not including them may lead to learning spurious correlations, such as temporal or article development biases. The neg_pre sampling adds useful information during training, despite including noise in the form of false negatives. Since our main goal is to predict promotional tone for a given text, we did not optimise for ranked prediction; however, the pairwise accuracy is of interest since the GRL-based model is trained on pairs. This is similar to the pairwise human evaluation we performed in Sec. 4. For this we calculate the proportion of pairs for which the directionality of the predictions is correct. A score of 0.722 is achieved for the non-GRL Longformer model, compared to 0.741 for the GRL model. The fact that these values are higher than the accuracy values in Table 3 illustrates that there are samples which were incorrectly classified, but whose relative (pair-wise) relationship was correctly predicted. To better understand the differences in predictions made by models trained with GRL, we analyse more closely the test set and our predictions. There are 1318 samples on which both models are correct and 764 on which both are incorrect. There are 320 samples where the non-GRL model is correct while the GRL model is incorrect, and 356 samples in the reverse case. We are interested in the last two categories, where the two models disagree. To better understand these classification categories, we evaluate the pointwise mutual information (PMI; Jurafsky and Martin, 2008) of each word 5607 ( w ) with its classification status ( c ) : P MI ( w, c ) = log 2 ( P ( w, c ) P ( w ) P ( c )) . This gives us an indication of how much higher the probability of observing a word is to be in one of the categories, compared to the full test set. The 50 words with the highest PMI, which were correctly classified as not promotional by our model but mislabeled as promotional by the non-GRL variant, are shown in Table 8 in Appendix D. Without GRL, these words were indicative of promotional tone; but with GRL, their use for promotional tone detection was reduced. Thus, these should be words that are misleading for the tone classifier, but helpful for the similarity classifier. The list includes the terms feminist, femi-nism and female. This topical concentration may be caused by a bias in the training data, whereby there are more positive examples which contain these terms. Such an imbalance in the data may be related to the findings of Wagner et al. (2016), which explores the imbalance in representations of women versus men on Wikipedia. However, this is not the only topical bias we observe in the predictions of the non-GRL model; the terms photograph, photographer, and graphics are also in this list. The PMI values for the opposite case where the non-GRL model is correct while the GRL model is incorrect is shown in Table 9. Here, too, we see some topical groupings; eg. tumor, physi-cians, diagnosis. However, the PMI of these words are lower than that of the samples where the GRL model was correct (with a maximum PMI of 3.92 vs 2.76), meaning that the co-occurrence is on the whole lower. We further evaluate our model on the test set from the per-article track of the SemEval 2019 Shared Task on Hyperpartisan News detection (Kiesel et al., 2019). Their dataset contains 314 positive (hyperpartisan) and 314 negative (not hyperpartisan) news articles. On this dataset, the Longformer+GRL model, trained on our training data, achieves a PR-AUC score of 0.759 (accuracy 0.785), compared to a PR-AUC of 0.736 (accuracy 0.714) when the GRL is omitted (statistically significant; P=0.043 on the signed rank test). The shared task received 42 entries and closed in June Test set No GRL Content Time FullTest 0.6936 0.6984 0.6901 ValidNegPre 0.5769 0.6184 0.5948 SemEval 0.6942 0.785 0.7531 Table 5: The results on each of the test scenarios from Sec. 7, comparing models with no auxiliary task, the content-based task we proposed, and the time-based auxiliary task. 2019. Compared against their leaderboard, our model would be ranked eighth, even though it was not trained on the provided training data. A motivation for including the neg_pre samples is that they counteract the temporal bias introduced by only sampling neg_post samples. The gradient reversal layer also provides a debiasing mechanism, used to suppress topic-based biases in our proposed model. To observe the impact of the neg_pre sampling, we also evaluate models trained with a time-based auxiliary task. Specifically, we define the task as predicting which sample in an input pair is earlier in the revision history of an article. We use only neg_post samples, as they were found to be less noisy. Samples are generated from ( neg_post , positive ) pairs as well as ( neg_post , neg_post ) pairs, with the chronological ordering being swapped at random to give an equal probability of both outcomes. The results on each of the test scenarios from Sec. 7 are shown in Table 5, using the Longformer feature encoder and comparing to the results from the original formulation. We also compare to the same model trained without an auxiliary task. The Time-GRL model outperforms the model with no auxiliary task on the ValidNegPre and SemEval datasets, but the content-GRL model scores the highest on all three test sets. This indicates that using neg_pre samples to counter temporal biases and the auxiliary task to counter content biases achieves better performance on this task. Previous work by Aleksandrova et al. (2019) explored a similar dataset creation strategy, using Wikipedia tags to identify sentences with a promotional tone. Our work focuses on document-level promotional tone detection, however, we also compare performance on our dataset using their models to verify whether document-level training captures more information than sentence-level training. We replicate their best performing model and the reported test set score of F1 score of 0.62. To compare performance on our own document-level data, we obtain a prediction for each sentence in an article and apply two aggregation strategies: using the average prediction and the maximum score. We further implement an LSTM model with attention, which is similar to our HAN model without the hierarchical computation. Results are shown in Table 6. In both cases, the mean aggregation yields a slightly better score; however, models trained on our data, both with and without the GRL optimisa-tion, achieve significantly higher scores, providing support that there is useful information contained in WikiEvolve for the task of document-level promotional tone detection. Finally, it worth noting that the LSTM+Attn model performs worse than the BoW+LogReg model. The authors also report comparatively worse performance for a model using FastText (Bo-janowski et al., 2017) embeddings. In this work, we have proposed an alternative data collection method and dataset for promotional tone detection, which leverages the evolution of articles on the platform. To utilise the additional structure in our dataset, we extended the gradient reversal framework to train models which are more effective at detecting promotional tone. This was shown both on our own test set and on a test set from a different domain. We further provided insights on the effects of two negative sampling strategies on Wikipedia. These findings should be useful for researchers who use Wikipedia-based data more broadly, in addition to those who work on biased language detection. Christine de Kock is supported by scholarships from Huawei and the Oppenheimer Memorial Trust. Andreas Vlachos is supported the EPSRC grant no. EP/T023414/1: Opening Up Minds. This work was initiated during an internship at the Wikimedia Foundation. We would like to thank the Foundation for granting us access to their data and resources, and in particular Diego Saez-Trumper for his support of the project.
abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain method objective abstain abstain abstain method abstain result abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method other abstain abstain abstain abstain abstain abstain other abstain abstain method abstain method abstain abstain method abstain abstain method method abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain objective objective result method abstain other other other other other