The advent of micro-blogging sites has paved the way for researchers to collect and analyze huge volumes of data in recent years. Twitter, being one of the leading social networking sites worldwide, provides a great opportunity to its users for expressing their states of mind via short messages which are called tweets. The urgency of identifying emotions and sentiments conveyed through tweets has led to several research works. It provides a great way to understand human psychology and impose a challenge to researchers to analyze their content easily. In this paper, we propose a novel use of a multi-channel convolutional neural architecture which can effectively use different emotion and sentiment indicators such as hashtags, emoticons and emojis that are present in the tweets and improve the performance of emotion and sentiment identification. We also investigate the incorporation of different lexical features in the neural network model and its effect on the emotion and sentiment identification task. We analyze our model on some standard datasets and compare its effectiveness with existing techniques. Social networking sites (e.g., Twitter) have become immensely popular in the last decade. User generated content (e.g., blog posts, statuses, tweets etc.) in social media provides a wide range of opinionated, emotional and sentimental content which gives researchers a massive data source to explore. For example, Twitter, being one of the leading social networking giants, provides an online environment that allows people of various backgrounds and locations to share their opinions and views on different matters. As of July 2018, over 500 million tweets are sent per day having over 300 million monthly active users. 1 1 http://www.internetlivestats.com/twitter-statistics/ There is often a misconception about sentiments and emotions as these subjectivity terms have been used interchangeably (Munezero et al., 2014). Munezero et al. (2014) differentiate these two terms along with other subjectivity terms and provide the computational linguistics community with clear concepts for effective analysis of text. While sentiment classification tasks deal with the polarity of a given text (whether a piece of text expresses positive, negative or neutral sentiment) and the intensity of the sentiment expressed, emotion mining tasks naturally deal with human emotions which in some end purposes are more desirable (Ren and Quan, 2012; Desmet and Hoste, 2013; Mohammad et al., 2015). Detecting emotion and sentiment from noisy twitter data is really challenging due to its nature. Tweets tend to be short in length and have a diverse vocabulary making them harder to analyze due to the limited contextual information they contain. In this study, we are interested in tackling these two tasks with a novel use of a single neural network architecture. A number of emotion theories are available which suggest different sets of basic emotions. Interestingly, joy, sadness, anger, fear and surprise are common to all. To the best of our knowledge, the model suggested by Ekman (1999) is the most broadly used emotion model. In this study, we use Ekman's basic emotions together with other sets of emotions (Plutchik, 1984; Shaver et al., 1987). In early textual emotion mining and sentiment analysis research, the usefulness of using external lexicons along with predefined rules has been demonstrated (Aman and Szpakowicz, 2008; Neviarouskaya et al., 2007; Bandhakavi et al., 2017; Thelwall et al., 2010; Gilbert, 2014). Aman and Szpakowicz (2008) used Roget's Thesaurus along with WordNet-Affect for fine-grained emotion prediction from blog data. Bandhakavi et al. (2017) propose a unigram mixture model (UMM)  K=1 K=2 K=3 K=1 Convolutional layer with multiple window sizes (k=1,2,3) Convolutional layer with single window size (k=1) Multichannel Embedding Layer Tweet Matrix (L 1 x d) Hash-Emo Matrix (L 2 x d) Dropout + Max Pooling to create a domain-specific lexicon which performs better in extracting features than Point-wise Mutual Information and supervised Latent Dirichlet Allocation methods. Neviarouskaya et al. (2007) propose a rule-based system which can handle informal texts in particular. They built a database of abbreviations, emoticons, affect words, etc., in which each entry is labeled with an emotion and its intensity. Thelwall et al. (2010) propose an algorithm, SentiStrength , which utilizes a dictionary of sentiment words associated with strength measures to deal with short informal texts from social media. Gilbert (2014) propose VADER , a rule-based model for sentiment analysis. They built a lexicon which is specially attuned to microblog-like contexts and their model outperforms individual human raters. More recently, deep learning models have proven to be very successful when applied on various text-related tasks (Kim, 2014; Kalchbrenner et al., 2014; dos Santos and Gatti, 2014; Tai et al., 2015; Wang et al., 2016; Felbo et al., 2017; Abdul-Mageed and Ungar, 2017). Kim (2014) showed the effectiveness of a simple CNN model that leverages pretrained word vectors for a sentence classification task. Kalchbrenner et al. (2014) propose a dynamic CNN model using a dynamic k-max pooling mechanism which is able to generate a feature graph which captures a variety of word relations. They showed the efficacy of their model by achieving high performances on binary and multi-class sentiment classification tasks without any feature engineering. dos Santos and Gatti (2014) propose a deep CNN model that uses both character and word-level information allowing them to achieve state-of-the-art performance on both binary and fine-grained multi-class sentiment classification for one of the twitter datasets. Tai et al. (2015) propose a Tree-LSTM model which captures syntactic properties in text. Their model performs particularly well on sentiment classification. Wang et al. (2016) propose a regional CNN-LSTM model for dimensional sentiment analysis. Their proposed model computes valence-arousal ratings from texts and outperforms several regression-based methods. Felbo et al. (2017) propose a bidirectional LSTM model with attention showing that their model learns better representations when distant supervision is expanded to a set of noisy labels. Abdul-Mageed and Ungar (2017) also used distant supervision to build a large twitter dataset and proposed a Gated Recurrent Neural Network model for fine-grained emotion detection. The recent success of neural based models motivated us to take a different look at the sentiment and emotion prediction from the noisy twitter data task. Compared with sequential models, CNN models train relatively faster and seem to work very well on noisy data such as tweets which are grammatically error-prone. We decided to work with CNN models after our initial experiments suggested that they perform comparatively better than a simple BiLSTM model on twitter dataset. We address the following questions in this paper:  Can CNN models be used in a way that can improve the performance of detecting emotion and sentiment from noisy Twitter data? How important are hashtag words, emoticons and emojis as predictors of emotion and sentiment in micro-blogging sites? How can we encode them in a multi-channel convolutional neural network? How can we add external features to a CNN model effectively? The remainder of the paper is organized as follows: We describe our model architecture in detail in Section 2. In Section 3, we describe the datasets and lexicons used in our experiments. As well, we describe the experimental setup required for working with Twitter data. In Section 4, we discuss the results from our experiments. In Section 5, we discuss our findings with particular attention paid to answering the above questions. Finally, in Section 6, we give a summary of our work followed by our remarks on future studies. We represent the architecture of our model in Fig. 1. The model consists of an embedding layer with two channels, a convolution layer with different kernel sizes and multiple filters, a dropout layer for regularization, a max pooling layer, multiple hidden layers and a softmax layer. We now describe each of these layers in detail. In this layer, two embedding matrices, the Tweet Matrix and the Hash-Emo Matrix, are passed through two different channels of our convolutional neural network. The first matrix represents a particular tweet. Each tweet t i consists of a sequence of tokens w 1 , w 2 , . . . , w n i . L 1 is the maximum tweet length. The height of the Tweet Matrix is L 1 . Short tweets are padded using zero padding. In the Tweet Matrix, every word is represented as a d -dimensional word vector. Since tweets are usually noisy, short in length, and have different kinds of features other than text, it's useful to have a word embedding specially trained on a large amount of Tweet data (Tang et al., 2014). Previous research (Collobert et al., 2011; Socher et al., 2011) has shown the usefulness of using pretrained word vectors to improve the performance of various models. As a result, in our experiments, we have used the publicly available pre-trained GloVe word vectors for Twitter by (Pennington et al., 2014a). The word vectors are trained on 27 B word tokens in an unsupervised manner. In this layer, we also pass another matrix called the Hash-Emo Matrix through a different channel in our network. This matrix is composed of three different sets of features: hashtags, emoticons and emojis. These are considered as distinguishable traits to showcase one's mood (Zhao et al., 2012). People like to use hashtags to express their emotional state through various micro-blogging sites (e.g., Twitter) (Qadir and Riloff, 2014). Also graphical emoticons or emojis can convey strong emotion or sentiment. So for each tweet t i , we extract hashtags h 1 , h 2 , . . . , h k i and emoticons/emojis e 1 , e 2 , . . . , e p i . We concatenate the hashtags and emoticon/emoji vectors for each tweet t i to get the Hash-Emo Matrix. We introduce a hyper-parameter L 2 as a threshold on the height of the Hash-Emo Matrix. Tweets with the number of hash-emo features less than L 2 are padded with zero while tweets with more hash-emo features than L 2 are truncated. We use word vectors from GloVe with dimension d for hashtags words. In the case that no word vector is found for a particular word we randomly initialize it. We also do random initialization of word vectors for emoticons. For emojis, we first map it to something descriptive (to be discussed in more detail in Section 3.2) and then generate random word vectors. These word vectors are tuned during the training phase. In this layer, we apply m filters of varying window sizes over the Tweet Matrix from the embedding layer. Here, window size ( k ) refers to the number of adjacent word vectors in the Tweet Matrix that are filtered together (when k > 1 ). Sliding our filter down we repeat this for the rest of the word vectors. Let w i  IR d be the d -dimensional word vector corresponding to the i -th word in a tweet. Also let w i  i + j denote the concatenation of word vectors w i , w i + 1 , . . . , w i + j and F  IR k  d denote the filter matrix. Thus a feature f i is generated by: f i = F  w i  i + k  1 + b (1) where b is a bias term and  represents the convolution action (a sum over element-wise multipli-cations). At this stage, we apply a nonlinear activation function such as ReLU (Nair and Hinton, 2010) before passing it through the dropout layer. We use multiple filters with the same window size in order to learn complementary features from the same window. Different window sizes ( k ) allow us to extract active local k -gram features. For the Hash-Emo Matrix, we apply m filters to each vector to generate local unigram features in different scales before passing it to the next layer. In this layer, employing a max-over pooling operation (Collobert et al., 2011) on the output from the previous layer for each channel extracts the most salient features. In this way, for each filter, we get the maximum value. So we get features equal to the number of filters in this stage. We chose max pooling instead of other pooling schemes because Zhang and Wallace (2017) showed that max pooling consistently performs better than other pooling strategies for various sentence classification tasks. We concatenate all the feature vectors from the previous layer. In addition, we concatenate additional sentiment and affect feature vectors (which are described in detail in Section 3.2) as well which forms a large feature vector. This is then passed through a number of hidden layers. A nonlinear activation function (i.e., ReLU (Nair and Hinton, 2010)) is applied in each layer before the vector is finally passed through the output layer. We tried a different activation function (tanh) as well, but ReLU worked the best for us. This is a fully connected layer which maps the inputs to a number of outputs corresponding to the number of classes we have. For multi-class classification task, we use softmax as the activation function and categorical cross-entropy as the loss function. The output of the softmax function is equivalent to a categorical probability distribution which generally indicates the probability that any of the classes are true. For binary classification task, we use sigmoid as the activation function and binary cross-entropy as our loss function. We used a number of emotion and sentiment datasets for our experiments. A description of each dataset is given below: BTD. Big Twitter Data is an emotion-labeled Twitter dataset provided by Wang et al. (2012). The dataset had been automatically annotated based on the seven emotion category seed words (Shaver et al., 1987) being a hashtag and the quality of the data was verified by two annotators as described in (Wang et al., 2012). We were only able to retrieve a portion of the original dataset as many tweets were either removed or not available at the time we fetched the data using the Twitter API. We applied the heuristics from (Wang et al., 2012) to remove any hashtags from the tweets which belong to the list of emotion seed words. TEC. Twitter Emotion Corpus has been published by Mohammad (2012) for research purposes. About 21,000 tweets were collected based on hashtags corresponding to Ekman's (1999) six basic emotions. The dataset has been used in related works (Shahraki and Zaiane, 2017; Balahur, 2013; Mohammad and Kiritchenko, 2015). CBET. The Cleaned Balanced Emotional Tweet dataset is provided by Shahraki and Zaiane (2017). To the best of our knowledge, this is one of the largest publically available balanced datasets for twitter emotion detection research. The dataset contains 80,937 tweets with nine emotion categories including Ekman's six basic emotions. SE. The SemEval-2018 Task 1 Affect dataset was provided by Mohammad et al. (2018). The SemEval task was to estimate the intensity of a given tweet and its corresponding emotion. However, in this study, we utilize the labeled dataset only to classify the tweets into four emotion categories and use the training, development and test sets provided in this dataset in our experiments. STS-Gold. This dataset was constructed by Saif et al. (2013) for Twitter sentiment analysis. The dataset contains a total of 2,034 tweets labeled (positive/negative) by three annotators. This dataset has been extensively used in several works for model evaluation (Saif et al., 2014b; Krouska et al., 2017; Saif et al., 2014a). STS. The Stanford Twitter Sentiment dataset was introduced by Go et al. (2009). It consists of a training set and a test set. The training set contains around 1.6 million tweets, whereas the test set contains 498 tweets. The training set was built automatically based on several emoticons as potential identifiers of sentiment. However, the test set was manually annotated and heavily used for model evaluation in related research. We perform one experiment with all three labels (posi-tive/negative/neutral) to compare the performance of different variants of our model and another one with two labels (positive/negative) to make comparison with related works (Jianqiang et al., 2018; dos Santos and Gatti, 2014; Go et al., 2009). SS-Twitter. The Sentiment Strength Twitter dataset was constructed by Thelwall et al. (2012) to evaluate SentiStrength. The tweets were manually labeled by multiple persons. Each tweet is assigned a number between 1 and 5 for both positive and negative sentiments, 1 represents weak sentiment strength and 5 represents strong sentiment strength. We followed the heuristics used by Saif et al. (2013) to obtain a single sentiment label for each tweet, giving us a total of 4 , 242 positive, negative and neutral tweets. The transformed dataset has been used in other literature (Go et al., 2009; Zhang et al., 2018). Data Cleaning. Twitter data is unstructured and highly informal (Yoon et al., 2013) and thus it requires a great deal of effort to make it suitable for any model. NLTK (Bird and Loper, 2004) provides a regular-expression based tok-enizer for Twitter, TweetTokenizer, which preserves user mentions, hashtags, urls, emoticons and emojis in particular. It also reduces the length of repeated characters to three (i.e. Haaaaaapy will become Haaapy). In our experiments, we utilized the TweetTokenizer to tokenize tweets. To accommodate the pretrained word vectors from (Pennington et al., 2014b), we pre-processed each tweet in a number of ways. We lowercased all the letters in the tweet. User mentions have been replaced with < user > token (i.e. @user-name1 will become < user > ). In addition, we also removed urls from the tweets as urls do not provide any emotional value. We also normalized certain negative words (e.g., won't will become will not). Using slang words is a very common practice in social media. We compiled a list of the most common slang words from various online resources 2 and replaced all of the occurrences with their full form (e.g., nvm will become never mind). Our list of slang words doesn't contain any word which has multiple meanings. Usage of certain punctuation is often crucial in social media posts as it helps the user to emphasize certain things. We found that two punctuation symbols (! and ?) are common among social media users to express certain emotional states. We kept these symbols in our text and normalized the repetitions (e.g., !!! will become ! < repeat > ). The use of emojis and emoticons has increased significantly with the advent of various social media sites. Emoticons (e.g., :-D) are essentially a combination of punctuation marks, letters and numbers used to create pictorial icons which generally display an emotion or sentiment. On the other hand, emojis are pictographs of faces, objects and symbols. The primary purpose of using emojis and emoticons is to convey certain emotions and sentiments (Dresner and Herring, 2010). One advantage of using the TweetTokenizer is that it gives us emoticons and emojis as tokens. Though we use the emoticons as is in our experiment, we utilize a python library called emoji to get descriptive details about the pictorial image. For example,  (cid:44)  represents smiling face . In our experiments, we removed stop-words and replaced numbers occurring in the tweets with the token < number > . We also stripped off  #  symbols from all the hashtags within the tweets (e.g., 2 Example: https://slangit.com/terms/social media Emotion Dataset BTD TEC CBET SemEval P R F1 P R F1 P R F1 P R F1 joy 68 . # depressed will become depressed) and used the stripped version of hashtags on the second channel of our model. We only kept tokens with more than one character. Input Features. Along with word embed-dings, we used additional affect and sentiment features in our network. In our experiments, we used a feature vector V f where each value in the vector corresponds to a particular lexical feature ranging between [ 0 , 1 ] . We utilized a number of publicly available lexicons which are described briefly below to construct the vector. Warriner et al. (2013) provides a lexicon consisting of 14,000 English lemmas with valence, arousal and dominance scores. Three components of emotion are scored for each word between 1 and 9 in this lexicon. We calculate the average score for each component across all tokens in a tweet and normalize them in the range [0, 1]. Gilbert (2014) provides a list of lexical features along with their associated sentiment intensity measures. We use this lexicon to calculate the average of positive, negative, and neutral scores over all the tokens in a tweet. In addition, we used the NRC Emotion Lexicon provided by Mohammad and Turney (2013) which consists of a list of unigrams and their association with one of the emotion categories (anger, anticipation, disgust, fear, joy, sadness, surprise, trust). We use the percentage of tokens belonging to each emotion category as features. We also used the NRC Affect Intensity Lexicon provided by Mohammad and Bravo-Marquez (2017) and NRC Hashtag Emotion Lexicon provided by Mohammad and Kiritchenko (2015) which contain real-valued fine-grained word-emotion association scores for words and hashtag words. We combined two lexicons MPQA and BingLiu provided by Wilson et al. (2005) and Hu and Liu (2004), respectively, and used them to calculate the percentage of positive and negative tokens belonging to each tweet. We also used AFINN (Nielsen, 2011) which contains a list of English words rated for valence with an integer between  5 (negative) to + 5 (positive). We first normalized the scores in the range [0,1] and then calculated the average of this score over all the tokens in a tweet. Lastly, we detect the presence of consecutive exclamation (!) and question marks (?) in a tweet and use them as boolean features. Network Parameters and Training. Zhang and Wallace (2017) performed a sensitivity analysis on various parameters of a one-layer CNN model and showed how tuning the parameters can affect the performance of a model. Inspired by the work done by (Zhang and Wallace, 2017), we also searched for the optimal parameter configurations in our network. Table 3 shows different hyper-parameter configurations that we tried and the final configuration that was used in our model. The final configuration was based on both performance and training time. The embedding dimension has been set to 100 for both channels of our network as it worked best for us among other dimensions. We also experimented with a different number of fil-Hyper-parameter Ranges Selected Embedding dimension 50 / 100 / 200 100 Number of filters 64 / 128 / 256 128 Kernel sizes 1 / 2 / 3 / 4 / 5 1 / 2 / 3 Batch size 16 / 30 / 50 16 Epochs 10 / 20 10 Dropout rate 0 . ters and varying kernel sizes. The combination of kernel sizes, ( k = 1 , 2 , 3 ) in the first channel and k = 1 in the second channel worked the best for us. We also experimented with various batch sizes and the performance of the model remained reasonably constant, though the training time varied significantly. In our network, we used three hidden layers. In addition, we used the Adam optimizer (Kingma and Ba, 2014) and the back-propagation (Rumelhart et al., 1986) algorithm for training our model. Keras 2.2.0 was used for implementing the model. Regularization. In order to reduce overfit-ting, it is a common practice to employ regularization strategies in CNNs. In our experiments, we used dropout regularization (Srivastava et al., 2014) for both of the channels after the convolutional layer. We experimented with three different dropout rates as seen in Table 3 and also with no dropout at all. The model works better when we apply dropouts after the convolutional layer. In this section, we describe the results obtained in our experiments. We use precision, recall, F1-score and accuracy as our evaluation metrics. In recent emotion category recognition studies on Twitter data, people tend to construct their own dataset by collecting tweets from Twitter for their experiments. Hence, it is hard to find a large enough benchmark dataset to compare the performance with other people's work. In this study, we experimented with four emotion labeled datasets which have been made publicly available by their authors. Table 2 shows the results for each emotion category for all of the datasets. For the BTD dataset, we trained our model with 1 , 136 , 305 tweets, while we used 140 , 979 and 142 , 602 tweets as development and test data respectively. We used the same training, development and test sets as (Wang et al., 2012) except that our retrieved dataset contains fewer samples. We achieved relatively high F1-scores of 72 . 6 % , 73 . 6 % and 76 . 8 % for joy, sadness and anger , respectively, whereas for surprise we get a low F1-score of 27 . 1 % . This is probably due to the imbalanced nature of the dataset as can be seen in Table 1. The number of samples for joy, sadness and anger is much higher than for surprise . Our model achieves an accuracy of 69 . 2 % , whereas Wang et al. (2012) reported an accuracy of 65 . 6 % when trained on a much larger dataset. We can not make direct comparison with (Wang et al., 2012) since we were not able to retrieve the full test set due to the unavailability of some tweets at the time of fetching data from Twitter. For the TEC dataset, we evaluated our model with 10-fold cross validation. Mohammad (2012) reported an F1-score of 49 . 9 % with SVM, whereas our model achieves an F1-score of 55 . 6 % . For the CBET dataset, we used 80% of the data as the training set and the remaining 20% as the test set. We get an average F1-score of 56 . 5 % . We also used 10-fold cross-validation for the SemEval dataset and achieved an F1-score of 61 . 3 % . Table 4 shows the performance of our model with 10-fold cross-Datasets Methods Positive Negative Neutral Accuracy P R F1 P R F1 P R F1 STS-Test CNN 73 . validation on different sentiment datasets with two classes (positive and negative). For the STS-Gold dataset, our model achieves an accuracy of 90 . 7 % whereas the previous best accuracy ( 86 . 0 % ) was reported by Jianqiang et al. (2018) with a deep CNN model. Our model achieves the best accuracy ( 90 . 3 % ) for the STS-Test dataset as well, while the previous best ( 87 . 6 % ) was reported in (Jianqiang et al., 2018). dos Santos and Gatti (2014) also experimented with the same dataset with their Character to Sentence CNN model, reporting an accuracy of 86 . 4 % . Lastly, for the SS-Twitter dataset, our model achieves an accuracy of 79 . 3 % whereas Zhang et al. (2018) and Saif et al. (2013) reported an accuracy of 61 . 9 % and 73 . 4 % , respectively. Tables 5 and 6 show the performance of three variants of our model on the sentiment labeled datasets and the emotion labeled datasets respectively. The first variant is a basic CNN model without hash-emo embedding or any additional features. The second variant includes the hash-emo embedding, while the last variant combines additional lexical features as well. It can be observed that when we introduce the second channel with hash-emo embedding, we get a significant increase in accuracy for most of the datasets. We can see in Table 5 that, for STS-Test and SS-Twitter datasets, we get better F1-scores for all three sentiment labels when we include the hash-emo embedding along with external lexical features. In Models Dataset BTD TEC CBET SE CNN 66 . Table 6, we can see that, inclusion of hash-emo embedding in the network gives us 2 . 4 , 3 . 3 , 2 . 3 and 3 . 5 percentage points increase in accuracy and the inclusion of additional features as well gives us 3 . 1 , 4 . 6 , 2 . 6 and 5 . 7 percentage points increase in accuracy for BTD, TEC, CBET and SE datasets, respectively, over the base models. In this study, we have showed the effectiveness of encoding hashtags, emoticons and emojis through a separate channel in a CNN network for emotion and sentiment detection tasks. Our MC-CNN model with hash-emo embedding performs well when compared to the basic CNN model. To the best of our knowledge, our model achieves the best accuracies on the three sentiment datasets, and has significant improvement in performance on the four emotion labeled datasets over the basic CNN model. The results show the importance of hashtags, emoticons and emojis in social media as predictors of emotion and sentiment. The model performs even better when additional lexical features are introduced into the network. In this paper, we propose a novel use of a multichannel convolutional neural architecture which effectively encodes different types of emotion indicators which are found in social media posts. Results suggest that encoding the emotion indicators through a separate channel provides significant improvement over the traditional CNN based models. We also demonstrate a simple approach to incorporate different lexical features in the network giving us comparatively better results when used along with our MC-CNN model. Our model performs particularly well on two important tasks in social media: emotion detection and sentiment analysis. This model can be extended to perform other tasks as well. this can give us noisy twitter data. We are thankful for the support by The Social Sciences and Humanities Research Council of Canada (SSHRC) through our Partnership Development Grant titled Living Archives of Rwandan Exiles and Survivors in Canada.
abstain abstain abstain abstain objective objective method abstain abstain abstain abstain other abstain abstain abstain abstain objective abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method result objective method method method method abstain result abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain method method method abstain method method abstain method method abstain method method abstain abstain abstain abstain method method abstain method other method method abstain method method method abstain abstain method abstain method method method method method method method method abstain abstain method method abstain method method abstain method method method method abstain abstain abstain method method method result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain result abstain result abstain result abstain abstain result abstain abstain abstain result abstain abstain result result abstain result abstain abstain abstain abstain abstain abstain abstain abstain result method result abstain abstain objective abstain objective method abstain abstain other