Learning word-pieces garble the predictions
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): Bert

Language I am using the model on (English, Chinese....): English

The tasks I am working on is:
* [ x] my own task or dataset: Sentiment Analysis on chatbot conversations

Word-pieces change the prediction completely, I fail to understand why.
If i do a token to token similarity also, by considering the token vector to be the average of wordpieces or the first token vector to see if they carry the same context, they don't.
If I had lemmatized the training set, these pieces would have not learnt anything really.
But that is not intuitive.

<img width="571" alt="Screenshot 2019-09-05 at 12 01 26 PM" src="https://user-images.githubusercontent.com/25073753/64317678-07a95680-cfd6-11e9-8849-f9c11f8531ae.png">
<img width="574" alt="Screenshot 2019-09-05 at 11 59 48 AM" src="https://user-images.githubusercontent.com/25073753/64317679-07a95680-cfd6-11e9-900a-96ec5d318b12.png">
<img width="594" alt="Screenshot 2019-09-05 at 11 59 53 AM" src="https://user-images.githubusercontent.com/25073753/64317680-07a95680-cfd6-11e9-9950-2957c284e6e8.png">

[CLS] token vector is fed to the classifier. Changes need to be done internally.
How can I handle such cases?
Any leads would be helpful.
Thanks in advance.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
