The inference speed of gpt2-xl has a gap between pytorch and tensorflow.
**Environment:**
- OS: Ubuntu 18.04
- Python: 3.7.6
- Transformers: 3.0.0
- PyTorch: 1.4.0
- Tensorflow: 2.2.0
- CUDA: 10.1
- CUDNN: 7.6
- GPU: V100

**My code:**
```import time
import torch
import tensorflow as tf
from transformers import AutoTokenizer, AutoModelWithLMHead, TFAutoModelWithLMHead

TIMES = 100

tokenizer = AutoTokenizer.from_pretrained("./gpt2-xl")

# pytorch
model = AutoModelWithLMHead.from_pretrained("./gpt2-xl")
model = model.to("cuda")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="pt").to("cuda")
total = 0
cnt = 0
start = torch.cuda.Event(enable_timing=True)
end = torch.cuda.Event(enable_timing=True)
for i in range(TIMES):
    start.record()
    o = model(input)
    end.record()
    torch.cuda.synchronize()
    if not i:
        continue
    total += start.elapsed_time(end)/1000
    cnt += 1
print("Pytorch version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))

# tensorflow
gpus = tf.config.experimental.list_logical_devices('GPU')
gpu = gpus[0].name
with tf.device(gpu):
    model = TFAutoModelWithLMHead.from_pretrained("./gpt2-xl")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="tf")
total = 0
cnt = 0
with tf.device(gpu):
    for i in range(TIMES):
        start = time.time()
        o = model(input)
        end = time.time()
        if not i:
            continue
        total += (end-start)
        cnt += 1
print("Tensorflow version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))
```

**Output:**
```
Pytorch version --- cnt: 99, avg_time_cost: 0.05521844493981564s
Tensorflow version --- cnt: 99, avg_time_cost: 0.2912752628326416s
```

**The utilization of gpu:**
- PyTorch 33%
- Tensorflow 8%

I'm new to Transformers. Did i use it in a wrong way, or any mistakes in my code?
Thanks!
Turning off the eager mode of tensorflow makes inference much faster.