Tweets are short messages that often include specialized language such as hashtags and emojis. In this paper, we present a simple strategy to process emojis: replace them with their natural language description and use pretrained word embeddings as normally done with standard words. We show that this strategy is more effective than using pretrained emoji embeddings for tweet classification. Specifically, we obtain new state-of-the-art results in irony detection and sentiment analysis despite our neural network is simpler than previous proposals. Tweets are short messages shared on Twitter, one of the most popular social networking services with 326 million monthly active users word wide (Twitter, 2018). Tweets often use specialized language such as abbreviations (e.g., TBH: To be honest ), hashtags (e.g., #NBAFinals ), emoticons and emojis. The Oxford Dictionary defines an emoticon as a facial expression such as a smile or frown, formed by various combinations of keyboard characters (e.g., :), :-(), and an emoji as a small digital image or icon used to express an idea or emotion (e.g., , , ). While the number of emoticons is relatively small, the Uni-code Standard includes over 2,800 emojis. Emojis are interesting because they succinctly encode meaning that otherwise would require more than one word to convey (e.g., grinning face , clapping hands and face with medical mask for the emojis above). Additionally, emojis have become popular in social media. 5 billion emojis are sent daily on Facebook (Burge, 2018). While only 6% of the top-100 Facebook headlines used emojis in 2015, 52% did so in 2017 (Boland, 2017). Over Irony? 14% of tweets and 50% of Instagram posts contain at least one emoji (Cruse, 2015; Moon, 2015). Irony detection and sentiment analysis in tweets are two popular tasks. Sentiment analysis has received substantially more attention than irony detection. Irony, however, is a major error source in sentiment analysis (0.71 F1 overall but 0.29 F1 with ironic tweets (Hee et al., 2018)), and natural language understanding in general does not generalize well with ironic texts (Liu et al., 2012; Maynard and Greenwood, 2014). In this paper, we tackle both irony and sentiment analysis in tweetstwo classification tasks. In particular, we focus on modeling emojis. Consider the examples in Table 1. Understanding the emojis is critical to making irony and sentiment judgements. In the first example, the contrast between the emojis helps determining that irony is present (the hashtag #not also helps). In the second tweet, the OK hand sign and face blowing a kiss emojis help reinforcing that the author is praising somebody and not being ironic. Similarly, the smiling and sad emojis in the last two examples are a clear sign of the author's sentiment towards the movie Ted 2 and the incompatibility issue. The main contributions of this paper are twofold. First, we present a simple strategy to model emojis: replace them with their textual description. Second, we show that this strategy outperforms previous methods and yields a new state-of-the-art in two tweet classification tasks: irony detection and sentiment analysis. Irony is closely related to sarcasm. The Oxford Dictionary defines irony as The expression of one's meaning by using language that normally signifies the opposite, typically for humorous or emphatic effect, and sarcasm as The use of irony to mock or convey contempt. Given these defi-nitions, it is not surprising that many researchers do not distinguish between them (Maynard and Greenwood, 2014). The top-3 systems to detect irony are built with neural networks and pretrained word embeddings. Baziotis et al. (2018) build an ensemble of two stacks of BiLSTMs (word and character level) with attention. Wu et al. (2018) propose a BiLSTM and a multitask learning framework (hashtag, irony presence and irony type prediction), and complement the input text with sentiment features extracted from lexicons. Vu et al. (2018) propose a multilayer perceptron taking as input an embedding for the input text (average of word embeddings) as well as manually crafted lexical, syntactic, semantic and polarity features. Our strategy to incorporate emojis outperforms all of them (Table 3). Sentiment analysis in tweets has been studied for years (Nakov et al., 2013). At its core, it is the task of classifying a tweet into expressing positive, neutral or negative sentiment (Rosenthal et al., 2017). Initial systems were primarily based on sentiment lexicons and manually extracted features, but the state of the art uses neural networks and word embeddings. Baziotis et al. (2017) propose a stack of two BiLSTMs at the word level and do not use any lexicons. Cliche (2017) presents a CNN and BiLSTM ensemble and experiment with three pretrained embeddings. Rouvier (2017) also presents a CNN and BiLSTM ensemble but incorporates manually defined features (e.g., word presence in emotion lexicons, all-caps). The strategy presented here to incorporate emojis outperforms all these systems (Table 4). Within natural language processing and social media, emojis have received considerable attention. Barbieri et al. (2016) train emoji embeddings with word2vec and discover that the closest words are sound (e.g., : coffee, roasters, caffeine, latte). Eisner et al. (2016) propose a complementary approach to train emoji embeddings (Sec-tion 3). Emojis have also been used as labels for distant supervision to improve tweet classification (Felbo et al., 2017). The strategy presented here to incorporate emojis is simpler and more effective than previous ones, does not require additional pretraining or domain specific corpora, and can be used with any neural architecture that takes text as input without any modifications. Simply put, we replace emojis with their textual descriptions and leverage existing pretrained word embeddings. Neural networks that take as input text usually transform the input tokens into pretrained embeddings. When the input text are tweets, it is common to use embeddings pretrained with large collections of tweets as opposed to general purpose text (Li et al., 2017; Pennington et al., 2014). Emojis as Regular Tokens. The simplest option to incorporate emojis into a neural network is to consider them as any other token in the input text (Barbieri et al., 2016). This strategy relies on having seen enough instances of each emoji in the texts with which embeddings were pretrained otherwise the embeddings will not capture the semantics of emoji tokens properly. Emoji Embeddings. Another strategy is to use separate embeddings for emojis. Eisner et al. (2016) pretrain emoji embeddings using positive and negative (randomly sampled) emoji descriptions. Descriptions are transformed into a vector by adding the corresponding word2vec embeddings (Mikolov et al., 2013). Emoji embeddings are tuned quickly because only a positive and a negative description per emoji are considered. Our Strategy: Emoji Descriptions. Our strategy is simple: replace emojis with their textual descriptions. Effectively, this eliminates all emojis in the input and incorporates a rather detailed descriptionseveral tokensof the emojis (see examples in Table 2). Our rationale is as follows. First, lists of emojis and their textual descriptions Emoji Description Face with tears of joy Face blowing a kiss Grinning face with smiling eyes Relieved face Squinting face with tongue Sad but relieved face Angry face Loudly crying face Downcast face with sweat Anxious face with sweat Table 2: Emojis and their textual description. This material is based upon work supported by the National Science Foundation under Grants Nos. 1734730, 1832267 and 1845757. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The Titan Xp used for this research was donated by the NVIDIA Corporation.
abstain method result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain objective objective objective other other other other other other other abstain other other other other other other other other other other other other method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain other other other