use of special tokens in gpt2?
![Screenshot from 2019-05-31 23-36-39](https://user-images.githubusercontent.com/36623119/58725591-24455980-83fd-11e9-81c6-d35fa4370e8e.png)
i am actually new to this field , I am trying to use gpt2 for sequence classification task in which i am adding "<|endoftext|>" after each sequence and using  last hidden state for classification.  My doubt is what is the use of special tokens such as "<|endoftext|>" if the gpt2tokenizer does not recognize them?
thank you and any suggestion in classification task is appreciated!
 
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
