<p>Word including Hash tag need to be tokanised. It has a special meaning and given a context. The word by itself has a different meaning.</p>

CC BY-SA 4.0
2021-02-18T13:03:27.260Z
So have they been encoded during training by models such as GPT or BERT?
user1234
CC BY-SA 4.0
2021-02-16T16:03:53.620Z
2021-02-16T16:03:53.620Z
raghu
<p>I am exploring a potential NLP project. I was wondering what generally is done with the hashtags words (e.g. <code>#hello</code>). Are those words ignored? is the <code>#</code> removed and the word tokenised? Is it tokenised with the <code>#</code>?</p>

<natural-language-processing><recurrent-neural-networks><word-embedding>
NLP: Are hashtags tokenised?
user1234
# NLP: Are hashtags tokenised?

(Asked by: username_0 on 2021-02-16)

I am exploring a potential NLP project. I was wondering what generally is done with the hashtags words (e.g. `#hello`). Are those words ignored? is the `#` removed and the word tokenised? Is it tokenised with the `#`?

**Tags:** natural-language-processing, recurrent-neural-networks, word-embedding


---

## 1 answers

### Answer by username_1 on 2021-02-16. Score: 0 | Accepted answer

Word including Hash tag need to be tokanised. It has a special meaning and given a context. The word by itself has a different meaning.

> **Comments:**
> **username_0**: So have they been encoded during training by models such as GPT or BERT?  

---

