Automatic hashtag annotation plays an important role in content understanding for microblog posts. To date, progress made in this field has been restricted to phrase selection from limited candidates, or word-level hashtag discovery using topic models. Different from previous work considering hashtags to be inseparable, our work is the first effort to annotate hashtags with a novel sequence generation framework via viewing the hashtag as a short sequence of words. Moreover, to address the data sparsity issue in processing short microblog posts, we propose to jointly model the target posts and the conversation contexts initiated by them with bidirectional attention. Extensive experimental results on two large-scale datasets, newly collected from English Twitter and Chinese Weibo, show that our model significantly outperforms state-of-the-art models based on classification. 1 Further studies demonstrate our ability to effectively generate rare and even unseen hashtags, which is however not possible for most existing methods. Microblogs have become an essential outlet for individuals to voice opinions and exchange information. Millions of user-generated messages are produced every day, far outpacing the human be-ing's reading and understanding capacity. As a result, the current decade has witnessed the increasing demand for effectively discovering gist information from large microblog texts. To identify the key content of a microblog post, hashtags, user-generated labels prefixed with a  #  (such as  #NAACL  and  #DeepLearning ), have been * This work was mainly done when Yue Wang was an intern at Tencent AI Lab. Target post for hashtag generation This Azarenka woman needs a talking to from the umpire her weird noises are totes inappropes professionally. #AusOpen Replying messages forming a conversation [T1] How annoying is she. I just worked out what she sounds like one of those turbo charged cars when they change gear or speed. [T2] On the topic of noises, I was at the Nadal-Tomic game last night and I loved how quiet Tomic was compared to Nadal . [T3] He seems to have a shitload of talent and the postmatch press conf. He showed a lot of maturity and he seems nice. [T4] Tomic has a fantastic tennis brain... widely used to reflect keyphrases (Zhang et al., 2016, 2018) or topics (Yan et al., 2013; Hong et al., 2012; Li et al., 2016). Hashtags can further ben-efit downstream applications, such as microblog search (Efron, 2010; Bansal et al., 2015), summarization (Zhang et al., 2013; Chang et al., 2013), sentiment analysis (Davidov et al., 2010; Wang et al., 2011), and so forth. Despite the widespread use of hashtags, there are a large number of microblog messages without any user-provided hashtags. For example, less than 15 % tweets contain at least one hashtag (Wang et al., 2011; Khabiri et al., 2012). Consequently, for a multitude of posts without human-annotated hashtags, there exists a pressing need for automating the hashtag annotation process for them. Most previous work in this field focuses on extracting phrases from target posts (Zhang et al., 2016, 2018) or selecting candidates from a pre-defined list (Gong and Zhang, 2016; Huang et al., 2016; Zhang et al., 2017). However, hashtags usually appear in neither the target posts nor the given candidate list. The reasons are two folds. For one thing, microblogs allow large freedom for users to write whatever hashtags they like. For another, due to the wide range and rapid change of social media topics, a vast variety of hashtags can be daily created, making it impossible to be covered by a fixed candidate list. Prior research from another line employs topic models to generate topic words as hashtags (Gong et al., 2015; Zhang et al., 2016). These methods, ascribed to the limitation of most topic models, are nevertheless incapable of producing phrase-level hashtags. In this paper, we approach hashtag annotation from a novel sequence generation framework. In doing so, we enable phrase-level hashtags beyond the target posts or the given candidates to be created. Here, hashtags are first considered as a sequence of tokens (e.g.,  #DeepLearning  as  deep learning ). Then, built upon the success of sequence to sequence (seq2seq) model on language generation (Sutskever et al., 2014), we present a neural seq2seq model to generate hashtags in a word-by-word manner. To the best of our knowledge, we are the first to deal with hashtag annotation in sequence generation architecture . In processing microblog posts, one major challenge we might face is the limited features to be encoded. It is mostly caused by the data sparsity exhibited in short and informal microblog posts. 2 To illustrate such challenge, Table 1 displays a sample Twitter post tagged with  #Au-sOpen , referring to Australian Open tennis tournament. Only given the short post, it is difficult to understand why it is tagged with  #AusOpen , not to mention that neither  aus  nor  open  appear in the target post. In such a situation, how shall we generate hashtags for a post with limited words? To address the data sparsity challenge, we exploit conversations initiated by the target posts to enrich their contexts. Our approach is bene-fited from the nature that most messages in a conversation tend to focus on relevant topics. Content in conversations might hence provide contexts facilitating the understanding of the original post (Chang et al., 2013; Li et al., 2015). The effects of conversation contexts, useful on topic 2 For instance, the eligible length of a post on Twitter or Weibo is up to 140 characters. modeling (Li et al., 2016, 2018) and keyphrase extraction (Zhang et al., 2018), have never been explored on microblog hashtag generation. To show why conversation contexts are useful, we display in Table 1 a conversation snippet formed by some replies of the sample target post. As can be seen, key content words in the conversation (e.g.,  Nadal ,  Tomic , and  tennis ) are useful to reflect the relevance of the target post to the hashtag  #AusOpen , because Nadal and Tomic are both professional tennis players. Concretely, our model employs a dual encoder (i.e., two en-coders), one for the target post and the other for the conversation context, to capture the representations from the two sources . Furthermore, to capture their joint effects, we employ the bidirectional attention ( bi-attention ) (Seo et al., 2016) to explore the interactions between two encoders' outputs. Afterward, an attentive decoder is applied to generate the word sequence of the hashtag. In experiments, we construct two large-scale datasets, one from English platform Twitter and the other from Chinese Weibo. Experimental results based on both information retrieval and text summarization metrics show that our model generates hashtags closer to human-annotated ones than all the comparison models. For example, our model achieves 45 . 03 % ROUGE-1 F1 on Weibo, compared to 25 . 34 % given by the state-of-the-art classification-based method. Further comparisons with classification-based models show that our model, in a sequence generation framework, can better produce rare and even new hashtags. To summarize, our contributions are three-fold:  We are the first to approach microblog hashtag annotation with sequence generation architecture. To alleviate data sparsity, we enrich context for short target posts with their conversations and employ a bi-attention mechanism for capturing their interactions. In this section, we describe our framework shown in Figure 1. There are two major modules: a dual encoder to encode both target posts and their conversations with a bi-attention to explore their interactions, and a decoder to generate hashtags. Input and Output. Formally, given a target post x p formulated as word sequence (cid:104) x p 1 , x p 2 , ..., x p | x p | (cid:105) and its conversation context x c formulated as word sequence (cid:104) x c 1 , x c 2 , ..., x c | x c | (cid:105) , where | x p | and | x c | denote the number of words in the input target post and its conversation, respectively, our goal is to output a hashtag y represented by a word sequence (cid:104) y 1 , y 2 , ..., y | y | (cid:105) . For training instances tagged with multiple gold-standard hashtags, we copy the instances multiple times, each with one gold-standard hashtag following Meng et al. (2017). All the input target posts, their conversations, and the hashtags share the same vocabulary V . Dual Encoder. To capture representations from both target posts and conversation contexts, we design a dual encoder, composed of a post encoder and a conversation encoder, each taking the x p and x c as input, respectively. For the post encoder, we use a bidirectional gated recurrent unit (Bi-GRU) (Cho et al., 2014) to encode the target post x p , where its embeddings e ( x p ) are mapped into hidden states h p = (cid:104) h p 1 , h p 2 , ..., h p | x p | (cid:105) . Specifically, h pi = [  h pi ;  h pi ] is the concatenation of forward hidden state  h pi and backward hidden state  h pi for the i -th token:  h pi = GRU ( e ( x pi ) ,  h pi  1 ) , (1)  h pi = GRU ( e ( x pi ) ,  h pi +1 ) . (2) Likewise, the conversation encoder converts conversations into hidden states h c via another Bi-GRU. The dimensions of both h p and h c are d . Bi-attention. To further distill useful representations from our two encoders, we employ the bi-attention to explore the interactions between the target posts and their conversations. The adoption of bi-attention is inspired by Seo et al. (2016), where the bi-attention was applied to extract query-aware contexts for machine comprehension. Our intuition is that the content concerning the key points in target posts might have their relevant words frequently appearing in their conversation contexts, and vice versa. In general, such content can reflect what the target posts focus on and hence effectively indicate what hashtags should be generated. For instance, in Table 1, names of tennis players (e.g.,  Azarenka ,  Nadal , and  Tomic ) are mentioned many times in both target posts and their conversations, which reveals why the hashtag is  #AusOpen . To this end, we first put a post-aware attention on the conversation encoder with coefficients:  cij = exp( f score ( h pi , h cj )) (cid:80) | x c | j (cid:48) =1 exp( f score ( h pi , h cj (cid:48) )) , (3) where the alignment score function f score ( h pi , h cj ) = h pi W bi  att h cj captures the similarity of the i -th word in the target post and the j -th word in its conversation. Here W bi  att  R d  d is a weight matrix to be learned. Then, we compute a context vector r c conveying post-aware conversation representations, where the i -th value is defined as: r ci = | x c | (cid:88) j =1  cij h cj . Analogously, a conversation-aware attention on post encoder is used to capture the conversation-aware post representations as r p . Merge Layer. Next, to further fuse representations distilled by the bi-attention on each encoder, we design a merge layer, a multilayer perceptron (MLP) activated by hyperbolic function: v p = tanh( W p [ h p ; r c ] + b p ) , (5) v c = tanh( W c [ h c ; r p ] + b c ) , (6) where W p , W c  R d  2 d and b p , b c  R d are trainable parameters. Note that either v p or v c conveys the information from both posts and conversations, but with a different emphasis. Specifically, v p mainly retains the contexts of posts with the auxiliary information from conversations, while v c does the opposite. Finally, vectors v p and v c are concatenated and fed into the decoder for hashtag generation. Decoder. Given the representations v = [ v p ; v c ] produced by our dual encoder with bi-attention, we apply an attention-based GRU decoder to generate a word sequence y as the hashtag. The probability to generate the hashtag conditioned on a target post and its conversation is defined as: P r ( y | x p , x c ) = | y | (cid:89) t =1 P r ( y t | y <t , x p , x c ) , (7) where y <t refers to ( y 1 , y 2 , ..., y t  1 ) . Concretely, when generating the t -th word in hashtag, the decoder emits a hidden state vector s t  R d and puts a global attention over v . The attention aims to exploit indicative representations from the encoder outputs v and summarizes them into a context vector c t defined as: c t = | x p | + | x c | (cid:88) i =1  dti v i , (8)  dti = exp( g score ( s t , v i )) (cid:80) | x p | + | x c | i (cid:48) =1 exp( g score ( s t , v i (cid:48) ) , (9) where g score ( s t , v i ) = s t W att v i is another alignment function ( W att  R d  d ) to measure the similarity between s t and v i . Finally, we map the current hidden state s t of the decoder together with the context vector c t to a word distribution over the vocabulary V via: P r ( y t | y <t , x p , x c ) = softmax ( W v [ s t ; c t ]+ b v ) , (10) which reflects how likely a word to be the t -th word in the generated hashtag sequence. Here W v  RV  2 d and b v  RV are trainable weights. Learning and Inferring Hashtags. During the training stage, we apply stochastic gradient descent to minimize the loss function of our entire framework, which is defined as: L () =  N (cid:88) n =1 log( P r ( y n | x pn , x cn ; )) . Here N is the number of training instances and  denotes the set of all the learnable parameters. In hashtag inference, based on the produced word distribution at each time step, word selection is conducted using beam search. In doing so, we generate a ranking list of output hashtags, where the top K hashtags serve as our final output. Datasets and Statistic Analysis. Two large-scale experiment datasets are newly collected from popular microblog platforms: an English Twitter dataset and a Chinese Weibo dataset. The Twitter dataset was built based on the TREC 2011 microblog track. 3 To recover the conversations, we used Tweet Search API to fetch in-reply-to relations in a recursive way. The Weibo dataset was collected from January to August 2014 using Weibo Search API via searching messages with the trending queries 4 as keywords. For gold-standard hashtags, we take the user-annotated hashtags, appearing before or after a post, as the reference. 5 The statistics of our datasets are shown in Table 2. We randomly split both datasets into three subsets, where 80 %, 10 %, and 10 % of the data corresponds to training, development, and test sets, respectively. To further investigate how challenging our problem is, we show some statistics of the hashtags in Table 3 and the distributions of hashtag frequency in Figure 2. In Table 3, we observe 3 https://trec.nist.gov/data/tweets/ 4 http://open.weibo.com/wiki/Trends/ 5 Hashtags in the middle of a post are not considered here as they generally act as semantic elements (Zhang et al., 2016, 2018). the large size of hashtags in both datasets. Moreover, Figure 2 indicates that most hashtags only appear a few times. Given such a large and imbalanced hashtag space, hashtag selection from a candidate list, as many existing methods do, might not perform well. Table 3 also shows that only a small proportion of hashtags appearing in their posts, conversations, and either of them, making it inappropriate to directly extract words from the two sources to form hashtags. Preprocessing. For tokenization and word segmentation, we employed the tweet preprocessing toolkit released by Baziotis et al. (2017) for Twitter, and the Jieba toolkit 6 for Weibo. Then, for both Twitter and Weibo, we further take the following preprocessing steps: First, single-character hashtags were filtered out for not being meaningful. Second, generic tags, i.e., links, mentions (@username), and numbers, were replaced with URL MENTION, and DIGIT, respectively. Third, inappropriate replies (e.g., retweet-only messages) were removed, and the remainder were chronologically ordered to form a sequence as conversation contexts. Last, a vocabulary was maintained with the 30 K and 50 K most frequent words, for Twitter and Weibo, respectively. Comparisons. For experiment comparisons, we first consider a weak baseline RANDOM that randomly ranks hashtags seen from training data. Two unsupervised baselines are also considered, where words are ranked by latent topics induced with the latent Dirichlet allocation topic model (henceforth LDA), and by their TF-IDF scores (henceforth TF-IDF ). Here for TF-IDF scores, we consider the N -gram TF-IDF ( N  5 ). Besides, we compare with supervised models below:  EXTRACTOR : Following Zhang et al. (2018), we extract phrases from target posts as hashtags 6 https://pypi.python.org/pypi/jieba/ via sequence tagging and encode conversations with memory networks (Sukhbaatar et al., 2015). CLASSIFIER : We compare with the state-of-the-art model based on classification (Gong and Zhang, 2016), where hashtags are selected from candidates seen in training data. Here two versions of their classifier are considered, one only taking a target post as input (henceforth CLASSIFIER ( post only )) and the other taking the concatenation of a target post and its conversation as input (hence-forth CLASSIFIER ( post+conv )). GENERATOR : A seq2seq generator (hence-forth SEQ 2S EQ ) (Sutskever et al., 2014) is applied to generate hashtags given a target post. We also consider its variant augmented with copy mechanism (Gu et al., 2016) (henceforth SEQ 2S EQCOPY ), which has proven effective in keyphrase generation (Meng et al., 2017) and also takes the post as input. The proposed seq2seq with the bi-attention to encode both the post and its conversation is denoted as OUR MODEL for simplicity. Model Settings. We conduct model tunings on the development set based on grid search, where the hyper-parameters that give the lowest objective loss are selected. For the sequence generation models, the implementations are based on the OpenNMT framework (Klein et al., 2017). The word embeddings, with dimension set to 200 , are randomly initialized. For encoders, we employ two layers of Bi-GRU cells, and for decoders, one layer of GRU cell is used. The hidden size of all GRUs is set to 300 . In learning, we use the Adam optimizer (Kingma and Ba, 2014) with the learning rate initialized to 0 . 001 . We adopt the early-stop strategy: the learning rate decreases by a de-cay rate of 0 . 5 till either it is below 1 e  6 or the validation loss stops decreasing. The norm of gradients is rescaled to 1 if the L 2 -norm > 1 is observed. The dropout rate is 0 . 1 and the batch size is 64 . In inference, we set the beam-size to 20 and the maximum sequence length of a hashtag to 10 . For CLASSIFIER and EXTRACTOR , lacking publicly available codes, we reimplement the models using Keras. 7 Their results are reproduced in their original experiment settings. For LDA, we employ an open source toolkit lda. 8 Evaluation Metrics. Popular information re-trival evaluation metrics F1 scores at K (F1@K) 7 https://keras.io/ 8 https://pypi.org/project/lda/ Model Twitter Weibo F1@1 F1@5 MAP RG-1 RG-4 F1@1 F1@5 MAP RG-1 RG-4 Baselines RANDOM 0.37 0.63 0.89 0.56 0.16 0.43 0.67 0.97 2.14 1.13 LDA 0.13 0.25 0.35 0.60 -0.10 0.86 0.94 3.89 TF-IDF 0.02 0.02 0.03 0.54 0.14 0.85 0.73 1.30 8.04 4.29 EXTRACTOR 0.44 -1.14 0.14 2.53 -7.64 5.20 State of the arts CLASSIFIER ( post only ) 9.44 6.36 12.71 10.75 4.00 16.92 10.48 22.29 25.34 21.95 CLASSIFIER ( post+conv ) 8.54 6.28 12.10 10.00 2.47 17.25 11.03 23.11 25.16 22.09 GENERATORSSEQ 2S EQ 10.44 6.73 14.00 10.52 4.08 26.00 14.43 32.74 37.37 32.67 SEQ 2S EQ-COPY 10.63 6.87 14.21 12.05 4.36 25.29 14.10 31.63 37.58 32.69 OUR MODEL 12.29 * 8.29 * 15.94 * 13.73 * 4.45 31.96 * 17.39 * 38.79 * 45.03 * 39.73 * Table 4: Comparison results on Twitter and Weibo datasets (in %). and mean average precision (MAP) scores (Man-ning et al., 2008) are reported. Here, different K values are tested on F1@K and result in a similar trend, so only F1@1 and F1@5 are reported. MAP scores are also computed given the top 5 outputs. Besides, as we consider a hashtag as a sequence of words, ROUGE metrics for summarization evaluation (Lin, 2004) are also adopted. Here, we use ROUGE F1 for the top-ranked hashtag prediction computed by an open source toolkit pythonrouge, 9 with Porter stemmer used for English tweets. For Weibo posts, scores calculated at the Chinese character level following Li et al. (2018). We report the average scores for multiple gold-standard hashtags on ROUGE evaluation. In this section, we first report the main comparison results in Section 4.1, followed by an in-depth comparative study between classification and sequence generation models in Section 4.2. Further discussions are then presented to analyze our superiority and errors in Section 4.3. Table 4 reports the main comparison results. For CLASSIFIER , their outputs are ranked according to the logits after a softmax layer. For EXTRACTOR , it is unable to produce ranked hashtags and thus no results are reported for F1@5 and MAP. For LDA, as it cannot generate bigram hashtags, no results are presented for ROUGE-SU4. In general, we have the following observations: 9 https://github.com/tagucci/ pythonrouge  Hashtag annotation is more challenging for Twitter than Weibo. Generally, all models perform worse on Twitter measured by different metrics. The intrinsic reason is the essential language difference between English and Chinese microblogs. English allows higher freedom in writing, resulting in more variety in Twitter hashtags (e.g., abbreviations are prominent like  aus  in  #AusOpen ). For statistical reasons, Twitter hashtags are more likely to be absent in either posts or conversations (Table 3), and have a more severe imbalanced distribution (Figure 2). Topic models and extractive models are ineffective for hashtag annotation . The poor performance of all baseline models indicates that hashtag annotation is a challenging problem. LDA sometimes performs even worse than RANDOM due to its inability to produce phrase-level hashtags. For extractive models, both TF-IDF and EXTRACTOR fail to achieve good results. It is because most hashtags are absent in target posts, as we see in Table 3 that only 2 . 72 % hashtags on Twitter and 8 . 29 % on Weibo appear in target posts. This confirms that extractive models, relying on word selection from target posts, cannot well fit the hashtag annotation scenario. For the same reason, copy mechanism fails to bring noticeable improvements for the seq2seq generator on both datasets. Sequence generation models outperform other counterparts. When comparing GENERATORS with other models, we find the former uniformly achieve better results, showing the superiority to produce hashtags with sequence generation framework. Classification models, though as the state of the art, expose their inferiority as they select labels from the large and imbalanced hashtag space (reflected in Table 3 and Figure 2). Conversations are useful for hashtag generation. Among the sequence generation models, OUR MODEL achieves the best performance across all the metrics. The observation indicates the usefulness of bi-attention in exploiting the joint effects of target posts and their conversations, which further helps in identifying indicative features from both sources for hashtag generation. However, interestingly, incorporating conversations fails to boost the classification performance. The reason why OUR MODEL better exploits conversations than CLASSIFIER ( post+conv ) might be that we can attend the indicative features when decoding each word in the hashtag, which is however not possible for classification models (considering hashtags to be inseparable). From Table 4, we observe that the classifiers outperform topic models and extractive models by a large margin but exhibit generally worse results than sequence generation models. Here, we present a thorough study to compare hashtag classification and generation. Four models are selected for comparison: two classifiers, CLASSIFIER ( post only ) and CLASSIFIER ( post+conv ), and two sequence generation models, SEQ 2S EQ and OUR MODEL . Below, we explore how they perform to predict rare and new hashtags. Rare Hashtags. According to the hashtag distributions in Figure 2, we can see a large proportion of hashtags appearing only a few times in the data. To study how models perform to predict such hashtags, in Figure 3, we display their F1@1 scores in inferring hashtags with varying frequency. The lower F1 score on less frequent hashtags indicates the difficulty to yield rare hashtags. The reason probably comes from the overfit-ting issue caused by limited data to learn from. We also observe that sequence generation models achieve consistently better F1@1 scores on hashtags with varying sparsity degree, while classification models suffer from the label sparsity issue and obtain worse results. The better performance of the former might result from the word-by-word generation manner in hashtag generation, which enables the internal structure of hashtags (how words form a hashtag) to be exploited. New Hashtags. To further explore the extreme situation where hashtags are absent in the training set, we experiment to see how models perform in handling new hashtags. To this end, we additionally collect instances tagged with hashtags absent in training data and construct an external test set, with the same size as our original test set. Considering that classifiers will never predict unseen labels, to ensure comparable performance, we only adopt summarization metrics here for evaluation and report ROUGE-1 F1 scores in Table 5. As can be seen, creating unseen hashtags is a challenging task, where unsurprisingly, all models perform poorly on this task. Nevertheless, sequence generation models perform much better on both datasets, e.g., at least 6.5x improvements over classification models observed on Weibo dataset. For Twitter dataset, the improvements are not that large, which confirms again that hashtag annotation on Twitter is more difficult due to the noisier data characteristics. In particular, compared to SEQ 2 SEQ , OUR MODEL achieves an additional performance gain in producing new hashtags by leveraging conversations with the bi-attention. To further analyze our model, we conduct a quantitative ablation study, a qualitative case study, and an error analysis. We then discuss them in turn. Ablation Study. We report the ablation study results in Table 6 to examine the relative contributions of the target posts and the conversation contexts. To this end, our model is compared with its five variants below: SEQ 2S EQ ( post only ), SEQ 2S EQ ( conv only ), and SEQ 2S EQ ( post+conv ), using standard seq2seq to generate hashtags from their target posts, conversation contexts, and their concatenation, respectively; OUR MODEL ( post-att only ) and OUR MODEL ( conv-att only ), whose decoder only takes v p and v c defined in Eq. (5) and Eq. (6), respectively. The results show that solely encoding target posts is more effective than modeling the conversations alone, but exploring their joint effects can further boost the performance, especially combined with a bi-attention mechanism over them. Case Study. We further present a case study on the target post shown in Table 1, where the top five outputs of some comparison models are displayed in Table 7. As can be seen, only our model successfully generates  aus open , the gold standard. Particularly, it not only ranks the correct answer as the top prediction, but also outputs other semantically similar hashtags, e.g., sport-related terms like  bbc football ,  arsenal , and  murray . On the contrary, CLASSIFIER and SEQ 2S EQ tend to yield frequent hashtags, such as  just saying  and  jan 25 . Baseline models also perform poorly: LDA produces some common single word, and TF-IDF extracts phrases in the target post, where the gold-standard hashtag is however absent. aus open  matches the gold-standard hashtag. To analyze why our model obtains superior results in this case, we display the heatmap in Figure 4 to visualize our bi-attention weight matrix W bi  att . As we can see, bi-attention can identify the indicative word  Azarenka  in the target post, via highlighting its other pertinent words in conversations, e.g.,  Nadal  and  tennis . In doing so, salient words in both the post and its conversations can be unveiled, facilitating the correct hashtag  aus open  to be generated. Error Analysis. Taking a closer look at our outputs, we find that one type of major errors comes from the unmatched outputs with gold standards, even as a close guess. For example, our model predicts  super bowl  for a post tagged with  #steel-ers , a team in super bowl. In future work, the semantic similarity should be considered in hashtag evaluation. Another primary type of error is caused by the non-topic hashtags, such as  #fb  (indicating the messages forwarded from Face-book). Such non-topic hashtags cannot reflect any content information from target posts and should be distinguished from topic hashtags in the future. Our work mainly builds on two streams of previous work  microblog hashtag annotation and neural language generation. We are in the line of microblog hashtag annotation. Some prior work extracts phrases from target posts with sequence tagging models (Zhang et al., 2016, 2018). Another popular approach is to apply classifiers and select hashtags from a candidate list (Heymann et al., 2008; Weston et al., 2014; Sedhai and Sun, 2014; Gong and Zhang, 2016; Huang et al., 2016; Zhang et al., 2017). Unlike them, we generate hashtags with a language generation framework, where hashtags in neither the target posts nor the pre-defined candidate list can be created. Topic models are also widely applied to induce topic words as hashtags (Krestel et al., 2009; Ding et al., 2012; Godin et al., 2013; Gong et al., 2015; Zhang et al., 2016). However, these models are usually unable to produce phrase-level hashtags, which can be achieved by ours via generating hashtag word sequences with a decoder. Our work is also closely related to neural language generation, where the encoder-decoder framework (Sutskever et al., 2014) acts as a springboard for many sequence generation models. In particular, we are inspired by the keyphrase generation studies for scientific articles (Meng et al., 2017; Ye and Wang, 2018; Chen et al., 2018, 2019), incorporating word extraction and generation using a seq2seq model with copy mechanism. However, our hashtag generation task is inherently different from theirs. As we can see from Table 4, it is suboptimal to directly apply keyphrase generation models on our data. The reason mostly lies in the informal language style of microblog users in writing both target posts and their hashtags. To adapt our model on microblog data, we explore the effects of conversation contexts on hashtag generation, which has never been studied in any prior work before. We have presented a novel framework of hashtag generation via jointly modeling of target posts and conversation contexts. To this end, we have proposed a neural seq2seq model with bi-attention over a dual encoder for capturing indicative representations from the two sources. Experimental results on two newly collected datasets have demonstrated that our proposed model significantly outperforms existing state-of-the-art models. Further studies have shown that our model can effectively generate rare and even unseen hashtags. This work is supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (No. CUHK 14208815 and No. CUHK 14210717 of the General Research Fund). We thank NAACL reviewers for their insightful suggestions on various aspects of this work.
abstain abstain objective objective result objective abstain abstain abstain method abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method abstain method objective method abstain abstain abstain method abstain method abstain abstain abstain result abstain method abstain abstain method result result abstain abstain objective objective method abstain abstain abstain other method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain method abstain abstain abstain abstain method other abstain other method abstain abstain method abstain abstain method abstain abstain abstain method abstain method method method method other abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain method other method abstain abstain method method abstain method abstain abstain method abstain method abstain method abstain abstain abstain abstain method method abstain method abstain other abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result result abstain abstain abstain abstain result result abstain abstain result abstain abstain abstain abstain method abstain other other method other other abstain method abstain abstain other objective objective objective objective result other other