Confused about the time of forword
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:3.0.0
- Platform: ubuntu 1604
- Python version: 3.7
- PyTorch version (GPU?):1.2.0 gpu
- Tensorflow version (GPU?): none
- Using GPU in script?: none
- Using distributed or parallel set-up in script?:No

### Who can help
@LysandreJik

## Information

Model I am using Bert, spceifically roberta_chinese_clue_tiny

The problem arises when using:
BertModel.from_pretrained

The tasks I am working on is:
just forward

Question:
Why the time of same forword is different? How to make it same?

## To reproduce
Steps to reproduce the behavior:
```
from transformers import BertTokenizer, BertModel
import time

tokenizer = BertTokenizer.from_pretrained("clue/roberta_chinese_clue_tiny")

model = BertModel.from_pretrained("clue/roberta_chinese_clue_tiny")

inputs = tokenizer("testtest", return_tensors="pt")

time_start=time.time()
outputs = model(**inputs)
time_end=time.time()

time_start2=time.time()
outputs = model(**inputs)
time_end2=time.time()

print('totally cost',time_end-time_start)
print('totally cost2',time_end2-time_start2)
```

## Expected behavior
```
totally cost 0.2720155715942383
totally cost2 0.007731199264526367
```

I assume that that's how Python works, if you run the same thing again, the result will be cached. If you had provided different inputs, then the time would be the same. <|||||>> I assume that that's how Python works, if you run the same thing again, the result will be cached. If you had provided different inputs, then the time would be the same.

Thanks for reply，but when i have tried differnt input(same length sentense), the inference time of first forword is longer than second forword(even 100x), i guess the model init produce the time cost<|||||>The same length sentences can have a different number of tokens after tokenizing. You could rather randomly create `input_ids` of the same shape to test this reliabley.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.