Slight different output between transformers and pytorch-transformers
I am now working on a Chinese NER tagging task. I applied BertForTokenClassification. The original library I used is pytorch-transformers 1.2.0. Then I migrated to Tranformers 2.1.1. But I found the output is slightly different between two versions. See the pics below

![Screen Shot 2019-10-24 at 14 46 00](https://user-images.githubusercontent.com/6031166/67460665-2b9c2680-f66e-11e9-9371-d29246dedc9f.jpg)
![Screen Shot 2019-10-24 at 14 44 47](https://user-images.githubusercontent.com/6031166/67460670-2d65ea00-f66e-11e9-8728-d615247107b6.jpg)

I wondered what potentially caused this difference?


## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....):
BertForTokenClassification

Language I am using the model on (English, Chinese....):
Chinese

The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS:
* Python version:
* PyTorch version:
* PyTorch Transformers version (or branch):
* Using GPU ?
* Distributed of parallel setup ?
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

Maybe you didn't put the model in evaluation mode in one of the tests and the DropOut modules were not deactivated as such.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
