Adversarial Examples (AEs) generated by perturbing original training examples are useful in improving the robustness of Deep Learning (DL) based models. Most prior works generate AEs that are either unconscionable due to lexical errors or semantically and functionally deviant from original examples. In this paper, we present ReinforceBug , a reinforcement learning framework, that learns a policy that is transferable on unseen datasets and generates utility-preserving and transferable (on other models) AEs. Our experiments show that ReinforceBug is on average 10% more successful as compared to the state-of-the-art attack TextFooler. Moreover, the target models have on average 73.64% confidence in wrong prediction, the generated AEs preserve the functional equivalence and semantic similarity (83.38%) to their original counterparts, and are transferable on other models with an average success rate of 46%. Machine Learning (ML) models have attained remarkable success in several tasks such as classification and decision analytics. However, ML Models specifically Deep Learning (DL) based models, are often sensitive to Adversarial Examples (AEs). AEs consist of modified training data samples that preserve the intrinsic utilities of the ML solutions, but influence target classifier's predictions between original and modified inputs (Fass et al., 2019). Recent works (Fass et al., 2019; Li et al., 2020; Biggio and Roli, 2018) have demonstrated that (i) including AEs as a part of training data can enhance the robustness and generalization of the ML models, and (ii) these examples can be utilized to test the robustness of ML models and help understand their security vulnerabilities and limitations. Previous works on generating AEs have attained success in image (Biggio and Roli, 2018) and on a few conventional text classification tasks such as sentiment, text entailment and movie reviews (Li et al., 2018; Jin et al., 2020; Li et al., 2020). Nevertheless, generating AEs for discrete textual data is still a challenge (Jin et al., 2020). Utility Preservation Textual AEs require to satisfy task-specific constraints such as lexical rules (spellings or grammar), semantic similarity and functional equivalence to original examples. Yet, most of the current state-of-the-art methods do not satisfy these constraints, thereby generating imperceivable AEs for end-users (Zhang et al., 2020). A few recent works (Li et al., 2020; Wang et al., 2019; Li et al., 2018; Jin et al., 2020) have considered semantic similarity constraint, but other constraints have been barely explored (Jin et al., 2020). Knowledge Transferability Most prior works Alzantot et al. (2018); Jin et al. (2020); Wang et al. (2019) generate one to one example-specific AEs, i.e., for a given example x , they create an example x (cid:48) . Each instance x is considered independent in a corpus C , and no relationship between different instances in the corpus is assumed. Therefore, the knowledge gained by transforming an example to AE is limited to a single example and is not reused on other examples. This process is both time-consuming and may not generalize the identified vulnerabilities of the target model. Word Replacement Strategy Most prior works use a single word replacement strategy such as synonym substitution (Alzantot et al., 2018; Jin et al., 2020; Wang et al., 2019) or character perturbation (Gao et al., 2018) to generate AEs. This strategy has two main disadvantages: (i) the AEs generated by using a character-based replacement strategy, contains many spelling errors that result in unnatural text; and (ii) multiple word transformations with its synonym in a synonym-based replacement strategy affect the language's fluency, making it sound unnatural. For instance, Jin et al. (2020) generated an AE Jimmy Wales is a big, fucking idiot liar friggin nincompoop deception". This AE sounds unnatural and is grammatically incorrect. Handling Noisy Datasets Most prior works generated AEs for datasets such as Yelp (Yelp) and Fake News (Kaggle, 2018a) which do not contain many spelling mistakes or out-of-the-vocabulary words. However, human generated natural text is prone to lexical errors. For example, tweets usually contain informal language, misspellings and unknown words. Prior works, such as (Jin et al., 2020; Wang et al., 2019; Alzantot et al., 2018; Li et al., 2020) that considered synonym substitution strategy, cannot deal with such noisy dataset. We present ReinforceBug that addresses the aforementioned limitations of prior works. Our code is available at https://bit.ly/2QOZDMT . Our main contributions are summarized as follows: 1. We propose a reinforcement learning framework, ReinforceBug , that learns a policy to generate utility-preserving AEs under the black-box setting with no knowledge about the target model architecture or parameters. 2. We evaluate ReinforceBug on four state-of-the-art deep learning models Email spam, Twitter spam, Toxic content and Review polarity detection respectively. 3. We investigate the transferability of our learned policy on a new dataset. 4. We also examine the transferability of our generated AEs on three other state-of-the-art deep learning models. Adversarial attacks are extensively studied in computer vision domain (Biggio and Roli, 2018; Good-fellow et al., 2014). Early works in adversarial text attacks were inspired by Generative Adversarial Networks (GANs) (Wong, 2017; Zhao et al., 2018). Wong 2017 showed that GAN-based reinforcement learning algorithms become unstable after a few perturbations. Later, heuristic-based methods such as word removal (Ebrahimi et al., 2018), Out-Of-Vocabulary (OOV) words (Gao et al., 2018) and synonym replacement (Li et al., 2018; Jin et al., 2020; Alzantot et al., 2018) have been proposed. Among these studies, DeepWordBug (Gao et al., 2018) generates AEs by randomly transforming a word by OOV word in an example. This approach is practical in producing AEs efficiently; however, it generates AEs that can be detected by the end-user due to large proportion of lexical errors. An attack framework named TextBugger (Li et al., 2018) was proposed to generate adversarial samples using the multi-level approach. It identified important words for each example and replaced them with optimal bugs. The approach considered both character-level and word-level transformations. Another recent attack, TextFooler (Jin et al., 2020) generates utility-preserving AEs by replacing an important word in an example with its grammatically equivalent synonym. The study also evaluates the generated AEs against semantic similarity constraint. Definition 1 A Deep Neural Network (DNN) is a machine learning model that learns a function F : X  Y over training data which maps from input space X to a set of classes Y. F is then evaluated on testing data X (cid:48) and F predicts the output label y (cid:48) for x (cid:48)  X (cid:48) . Definition 2 Prediction Confidence Score PCS( x , y ) of model F depicts the likelihood of an input x  X having a label y  Y . The smaller PCS( x , y ) suggests F has low confidence that x has a label y . Definition 3 Given a real example x having a label y , a utility-preserving AE against x is an input x (cid:48)  x +  with a minor perturbation  such that x (cid:48) satisfies a set of perturbation constraints P const and F predicts an incorrect label for it with high PCS i.e., y (cid:48)  F ( x (cid:48) ) such that y (cid:48) (cid:54) = y and PCS( x (cid:48) , y (cid:48) ) >  . Definition 4 A black-box attack is an attack where an attacker does not know the target model F architecture, training data X or hyper-parameters  . An attacker can only query F with Figure 1: Overall of ReinforceBug the input examples X (cid:48) and gets the corresponding P CS ( x, y ) . Definition 5 A non-targeted attack is an attack in which the adversary's goal is to maximize the mis-classification rate of the model F on any generated AE irrespective of its ground-truth label y . i.e., fool the model F to misclassify a spam email as benign email or vice-versa. Given a pre-trained target model F , we need to simulate a non-targeted (Definition 5) black-box attack (Definition 4) (Morris et al., 2020) to generate a set of utility-preserving AEs (Definition 3) A exp from a Corpus C with N examples and having corresponding target label t g  Y . Furthermore, our approach should learn a policy   ( s,a ) to perform perturbation  on C such that the model generates utility-preserving AEs (Definition 3) that have semantic similarity ( >  ) with the original example but have low perturbation rate (number of words perturbed) and lexical (grammatical and spelling) errors. Moreover, the policy should be transferable on unseen datasets. Fig 1 provides an overview of ReinfoceBug. We model an attack as a reinforcement learning (Sutton and Barto, 2018) process consisting of three main components: an environment, Proximal Policy Optimization (PPO) agent (Schulman et al., 2017) and action space. Firstly, the environment state ( s t ) at time t is processed as input by an agent followed by an action a t  A determined by an agent to update s t to the next state s t+1 , where A represents an action space (set of valid actions given the state). Subsequently, the environment's actuator acts on the corpus state to construct candidate examples. These examples are then sent to the reward generator module, which is responsible for computing the reward r t + 1 for action a t . The reward generator applies post constraints P ost  P const , queries the target model F and obtain scores of the candidate examples to calculate the reward of action a t . The reward is sent back to the actuator which determines a valid update to the corpus state as well as the next environmental state s t+1 . The experience consisting of < s t , a t , r t+1 , s t+1 , va t +1 > is sent to the agent and the agent model is updated. Here va t +1 shows the valid actions mask for next state. The va t +1 is then used by the agent to update an action space A and restrict the agent to only select valid action on the new state s t +1 . Each of the modules is discussed below: 3.3.1 Agent We use a customized version of Proximal Proximate Optimal (PPO) (Tang et al., 2020) Reinforcement Learning (RL) agent with action mask capability. PPO is an enhanced version of Actor-Critic (AC) Model (Grondman et al., 2012) . In AC architecture the agent consists of a separate policy and Q-value network. They both take in environment state s t at each time step as an input, while actor determines an action a t among the possible valid action set A and critic yields value estimation V t ( s t ) . While an actor uses a gradient descent algorithm to learn a policy that maximizes the rewards R t , the critic learns to estimate R t via minimizing a temporal loss. Further, the PPO algorithm avoids large and inefficient policy updates by constraining the new policy updates to be as close to the original policy as possible. We have selected this agent because our action and state space is substantially large and to avoid enormous policy updates which make the agent unstable (please refer to Schulman et al. (2017) for more details). The environment takes action a t from the agent as input and outputs the experience tuple e t , AEs A exp and a flag done depicting the success of the agent in achieving the goal. 1. Corpus State ( C t ) : it is given by C t = E 1 , E 2 , ...., EN , where N is the number of examples in the corpus and E i is equivalent to the set of words W i = w 1 , w 2 , ..., w n for an example i at time t . 2. Score State ( score t ) : it is a vector representing the PCS (Definition 2) of target model F on examples E having a ground-truth label tg in corpus C at time t. For instance, given an example E i the score t [ i ] = P CS ( E i , tg ) where tg is a ground truth label of E i . 3. Environment State ( s t ) : this state s t = w 1 , w 2 , ..., w k is observable by an agent. It consists of k mutable important words in corpus C . 4. Success rate ( success t ) : it is the proportion of utility-preserving AEs compared to all AEs generated by the agent at time t . Word selector This component takes C state at t = 0 and pre-constraints P re  P const as inputs. P re are task-specific perturbation restrictions on the specific entities in E i . For example, spam messages mostly contain URLs, IP addresses, organization names and email addresses pointing to phishing websites. Perturbing these entities in the spam message can change the functional semantics of the message (Ranganayakulu and Chellappan, 2013). Hence, imposing pre-constraints ensures that generated AE preserves the functional equivalence to E i after applying perturbations. To achieve this, we have designed a countvectorizer (sklearn) using a customized tokenizer. The tokenizer finds the list of immutable entities such as URLs and IP addresses in the text using regular expressions or named entity models (Florian et al., 2003). After that, these words are segmented into immutable words Im words using word tokenizer and saved for each example to be utilized later by actuator module. For training ReinforceBug , our method first computes the important words from the training datasets as state ( s t ) and then learns a policy to identify best actions ( a t ) to transform the s t to a next state ( s t +1 ) such that the success of our attack is maximum. Our work extends Jin et al. (2020) important words component; however instead of generating example-specific words, our module identifies corpus-specific important words. An important word is selected using a word importance score I w idx . The I w idx is calculated as the sum of prediction change in all the examples (k) containing w idx before and after deleting w idx . The candidate words w idx  most frequent words in the training dataset vocabulary. It is formally defined as follow. If the I w idx is > 0 , we consider it as an important word. The final list of all the important words is considered as the state s t . s t and mapping C map that maps each word to the corpus examples are sent back to the environment. For testing, the designed countvectorizer is used to transform the testing data onto these selected words. Actuator This module is responsible to execute an action a t selected by an agent. Firstly, the actuator transforms action a t into an action tuple < w idx , act idx , rep idx > , where w idx , act idx and rep idx depict the index of the word to be replaced, the operation to be performed on that word and replacement word index respectively. Subsequently, example indexes E idx containing the word w idx are obtained by querying the C map . After that, the actuator examines the score t [ k ] (score state) of each example k in E idx . If score t [ k ] >  , only then it is selected as an example to be perturbed, here  represent the PCS threshold. In this way, the examples for which AEs have already been found are not perturbed further and other examples are given a chance. Once these examples are selected, the operation act idx is applied to w idx which results in multiple replacement options. Table 1 provides the list of operations considered. For example, if an operation is Homoglyph and the word indexed by w idx is solid" then following replacements can be done "so1id,sol1d,s0lid,5olid". The new word w new is selected by rep idx . After w new is selected against the w old , if w old in not in the immutable word list of example k than a candidate AEs are generated by substituting w old with w new in the previously selected examples. In this way the functional equivalence is ensured before generation of AE . The selected candidate examples are then sent to the reward generator. The reward generator module returns the reward of changing the w idx in the selected examples by applying act idx and selecting rep idx . The reward generator also outputs the AEs that satisfies all the post constraints. Finally, the state is updated by replacing the w idx in s t with index of new word w new . All further actions on w idx are then invalidated. This is done by setting the va t+1 for all actions on the w idx in an action _ space to False. In this way, multiple actions on the same word cannot be performed in one training episode. The success t +1 is updated, the episode completes if the success t +1 for the corpus state C t+1 has reached the threshold specified by  or all the words in the state have been updated. The module constructs an experience tuple e t for the agent and returns e t , C t+1 , score t+1 and success t+1 to the environment. Reward Generator (RG) Algorithm 1 shows the pseudo-code of the RG module. RG takes candidate examples, C t , time t and post constraints as input and outputs the Adv exp , the reward r t+1 of a t and updated corpus state C t +1 . In this study, we have considered three main post utilities that the generated Adv exp should preserve apart from ALGORITHM 1: Reward_Generator 1 Input: Original examples org , candidate examples cand , time t , Post  P const ; 2 Output: Adv exp , reward r t +1 , C t+1 ; 3 Initialize reward  0 , Adv exp  {} ; 4 updated_cand, rewards; 5 if ( t = 0 ) then 6 updated _ cand = cand ; 7 Query Target Model ; 8 score t  target _ model.query ( cand ) ; 9 else 10 Initialize thresholds for spellerror  , gramerror  , Semantic   Post ; 11 N (cid:48)  total candidate examples; 12 forall c k  cand do 13 sem = Semantic ( org k , c k ) ; 14 spell = spellerror ( c k ) wordlen ; 15 gram = gramerror ( c k ) wordlen ; 16 if sem >  and spell < =  and grammar < =  ) then 17 Query target_model ; 18 new _ sc k = query ( c k ) prev _ sc k  score t [ k ] ; 19 change [ k ] = prev _ sc k  new _ sc k prev _ sc k ; 20 if change [ k ] > 0 then 21 update the corpus state ; 22 C t+1  E idx [ k ]  c k ; 23 update the score state ; 24 score t+1 [ k ]  new _ sc k ; 25 end 26 Compute Reward ; 27 if ( score t [ k ] <  ) then 28 Adv exp changing the output of the classifier. Firstly, the percentage of the spelling and grammatical errors should not be more than  and  respectively, and the semantic similarity between the original and AE should be >  . If the candidate example meets all these utilities, then the target _ model is queried to obtain the score of the candidate example. RG updates the corpus and score states for an example where the difference between previous (before perturbation) and new (after perturbation) score of an example > 0 . Subsequently, the reward generator checks if the candidate example score has converged to <  or not. When converged case, the example is added to the list of valid AE against the original example, and a reward r t +1 is computed as shown in line 30 of Algorithm 1. Otherwise, the sum of the change in the scores of an example is added as a reward, as shown in line 31. In line 30, the r t +1 represents the summation of reward attained on successfully transforming the original examples into well-constrained AE. It is calculated as a summation of the change in original and current score of the example k, assisted by semantic similarity with the original example and penalized by lexical errors (spelling and grammatical) and perturbation rate p rate for generating each AE. In this way, the agent learns to generate AEs with minimum perturbations and lexical errors and more PCS and semantic similarity. Lastly, the r t +1 is normalized by the factor N', so that the agent can learn the im-pact of action a t on a single example irrespective of the size of the corpus. Target Model We have considered a black-box attack (Definition 4) against the target model F . F can be any DNN (Definition 1) that provides P CS (Definition 2) score as an output. This section presents our experiment details. We studied the effectiveness of ReinforceBug on three noisy and one conventional text classification tasks respectively. Table 2 enlists the statistics of the considered datasets. Precisely during the target model training, we held out 30% of the training data as a validation set, and all parameters were tuned based on it. After that, the testing dataset was used to evaluate the performance of the model. For training and testing our ReinforceBug , we split the dataset into training (70%) and testing dataset (30%). The stratified split was applied to ensure the class distribution on these datasets remains consistent with the actual testing dataset. For each dataset, we trained four state-of-the-art models namely Word Convolution Neural Network (CNN) (Jain et al., 2018), Character CNN (Zhang et al., 2015), Word Bidirectional Long Term Short Memory (BiLSTM) (Zhou et al., 2016) and Recurrent CNN (Lai et al., 2015) on the training set. However, we did not consider the recent state-of-the-art BERT (Devlin et al., 2019) model because we found that its performance was significantly low for our considered noisy datasets. Srivastava et al. 2020 also reported that the BERT model's performance notably degrades for noisy text datasets, and further research is required to fine-tune BERT for noisy datasets. For training the considered models, we used an open-source GitHub repository (Lee). Table 3 shows the performance of each model on the testing set. From these models, we selected the models with best performance accuracy as target models (as highlighted in bold) to train ReinforceBug . For the rest of the models trained on a similar dataset, we studied the transferability of our generated AEs. Moreover, we tested our ReinforceBug against an unseen dataset to study the transferability of our attack on other datasets. Lastly, for get _ semantic and get _ synonym operations (Pennington et al., 2014) and (Mrkic et al., 2016) embeddings had been used. We used stable-baselines (Hill et al., 2018) reinforcement learning library to implement our PPO agent. We used a Multilayer perception (MLP) model, as an actor and critic models. For training the agent, we used 30 episodes for each model. To ensure the functional equivalence, we defined immutable tokens as pre-constraints using Named Entity Model provided by Spacy and regular expressions. For Enron dataset, names (Person or Organization), IP, email addresses and URL, while for Twitter and Toxic datasets URL, #Hashtags and @Reference and lastly, for Yelp dataset, names and URL were considered as immutable entities. For Table 2: The Target Models training and testing data sets statistics Dataset Training Testing Avg Avg SpellAvg GramData Data Length ing errors mar errors Enron (Wiki) 28.6k 9.9k 244 3.07% 31% Twitter (Kaggle, 2019) 8.1k 3.9k 15 10.67% 28.46% Toxic (Kaggle, 2018b) 159.5k 12.4k 70 2.13% 25.60% Yelp (Yelp) 560k 38k 139 0.81% 21.67% Table 3: Balanced Accuracy of target models on test datasets Dataset WordCNN CharCNN BiLSTM RCNN Enron 97.50% 96.50% 97.60% 98.30 % Twitter 93.44% 92.02% 93.72% 94.15 % Toxic 69.05% 86.62% 89.61 % 88.56% Yelp 94.74% 94.44% 95.60 % 95.34% semantic and lexical equivalence, we defined three post constraints, i.e., the semantic similarity, which were calculated using Universal Sentence Encoder (USE) (Cer et al., 2018) considering  = 0 . 60 . Moreover, for counting spelling mistakes Garbe was used while for calculating grammar issues, we used language tool (PYPI). Results of our experiments are presented here. Table 4 shows the samples of AEs generated by ReinforceBug and Table 5 illustrates the main results of our experiments. It can be seen that ReinforceBug produces AEs with a comparatively high PCS (i.e., on average 74%), semantic similarity (i.e., on average 83.5%) than other two attacks for all the datasets. However, its success rate is on average 15% less than TextBugger (Li et al., 2018) for the all the models. One reason behind it is that TextBugger generates unrealistic AEs with least PCS (i.e., on average 66%), low semantical similarity (i.e., on average 63%) and significantly high lexical errors (i.e., on average 14% spelling and 27% grammar errors). Additionally, TextBugger on average perturbs more than 59% words of the original text to generate AEs. Such a high perturbation rate is too large to be ignored by the end-user. Moreover, for Enron and Twitter datasets, TextBugger perturbs 9.09% and 97.67% of the URLs present in the text, thus adversely effecting the functional semantics of the text. Therefore, the success rate of TextBugger is an overestimation and deviates from the semantic, functional and lexical constraints. In comparison with TextFooler (Jin et al., 2020) our method has significantly high success rate (i.e., on average more than 10%) for all the models. It is expected because TextFooler relies on synonym substitutes technique to generate AE, however, for noisy datasets with relative high lexical errors such as Twitter (Table 2) this method tends to fail. Lastly, TextFooler produces 5% more grammatical errors than ReinforceBug and similar to TextBugger, TextFooler also perturbs URLs, thus effecting the functional semantics of the generated AEs. These findings suggest that ReinforceBug produces effective and utility-preserving AEs (Def-inition 3). Table 6 shows the transferability of policy learned by ReinforceBug on unseen datasets for each task. For benchmarking the results are also compared with the state-of-the-art attacks TextBugger and Textfooler . It is evident from the results that ReinforceBug takes less time to generate adversarial samples as compared to the other models. It is because ReinforceBug utilizes the same important word vocabulary selected while learning and the agent has already explored and learned utility-preserving operations on them during training. Additionally, although TextFooler and TextBugger both perform example-specific perturbation and ReinforceBug might suffer from out-of-vocabulary words but still the success rate of ReinforceBug on test datasets is more than TextFooler attack and less than TextBugger which are aligned with our finding in section 5.1. Also, from Table 6 it is evident that our ReinforceBug produces utility-preserving AEs with high PCS and semantic similarity (i.e., on average more than 70% and 82% respectively) and comparatively low perturbation rate (on average 8.5%) for all test datasets. It suggests that the important words and transformation learned from the training datasets are transferable to unseen datasets and can be used to generalize the vulnerabilities of the target models. To determine whether AEs curated based on one model can also fool other models for the same task, we examined the transferability of AEs on other models (see Table 3 for the performance of these models). Table 7 shows the transferability result. AEs generated by our method are more transferable to other models in comparison to the state-of-the-art attacks. However, there is a moderate degree of transferability between models, and the transferability is higher for Twitter and Toxic detection task as compared to Email and Yelp movie-review classification task. Nevertheless, BiLSTM trained on Enron dataset (having 97.60% accuracy) offers more resilience to AEs generated by RCNN by limiting the success rate of the attack to (<17%) for all the attacks, while other models are highly vulnerable to the AEs. It signifies that vulnerabilities exploited by our AEs are task-specific and moderately model-independent. Figure 2a illustrates the proportion of each operation chosen by the ReinforceBug to generate utility-preserving AEs. We can see that get _ semantic , and get _ synonyms operations are most dominant for all the tasks. One reason could be that get _ semantic is deliberately designed for creating similar contextual adversarial texts without deleting the original important word, while get _ synonyms replaces the word with similar meaning word. That is why the semantic similarity remains intact without impacting the linguistic structure of the text. Other operations, i.e., addition , insertion , bitsquatting and omission that cause common typos are moderately chosen, however, generatehom is rarely selected by ReinforceBug . This happens because it can only replace a character with a visually similar character and produces fewer replacement options for a word than other operations. This reason is also valid of word _ month and word _ to _ num as only few words are either words representing month or numbers in the corpus vocabulary. To demonstrate the knowledge transferability, we visualize the identified important words according to the number of examples affected by their replacements in Figure 2b. Here, the words impacting more examples are represented with a larger font. Figure 2b(i) shows that in emails, words such as click', attached', thanks', and deal' are more likely to affect the prediction of the target models by decreasing the spam intent to benign. Whereas for the Twitter dataset, Figure 2b(ii) shows that the targeted RCNN model is more vulnerable to minor perturbation on words such as news', trump', obama', state' and police'. For toxic content detection, the model decision is manipulated for most examples with words like see', like', think'. Perturbing words like stupid' and idiot' decreases the toxicity of the text. Lastly, for the yelp dataset, changing words such as like', great' and best' increases the text's negative extent. Therefore, it is evident that models are vulnerable to these words irrespective of a specific example; instead, these vulnerabilities affect multiple examples in the corpus and are transferable to new datasets as seen in (section 5.2) as well as are transferable to other models (section 5.3). Overall, this study proposes ReinforceBug , a reinforcement learning-based framework to generate utility-preserving AEs against the state-of-the-art text classifiers under black-box settings. Extensive experiments demonstrate that it effectively generates utility-preserving AEs that are transferable to other models, and the learned policy is transferable to the unseen datasets. It identifies semantic concatenation and synonym substitution attacks as a significant threat to text-classifiers and suggests defences against these attacks should be explored in the future to improve their robustness. This work was supported with super-computing resources provided by the Phoenix HPC service at the University of Adelaide.
abstain abstain method result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain abstain method abstain method other other other other other other other other other other other abstain abstain abstain other abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain method abstain method abstain method method abstain method abstain method method method abstain method method method method abstain method abstain method result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain other