Wrong shape output for loss of TFGPT2LMHeadModel
## Environment info
     
- `transformers` version: 4.0.0
- Platform: Ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): N/A
- Tensorflow version (GPU?): 2.3
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik
@jplu

## Information

Model I am using (Bert, XLNet ...): TFGPT2LMHeadModel

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

The following in a Python interpreter:
```python
import transformers
model = transformers.models.gpt2.TFGPT2LMHeadModel.from_pretrained('gpt2')
input_ids = tf.ones((1, 1024), dtype=tf.int32)
labels = tf.ones((1, 1024), dtype=tf.int32)
print(model(input_ids, labels=labels, return_dict=True, training=True).loss.shape)
```
Outputs
```
TensorShape([1023])
```

It seems the loss output is dependent on batch size: 
```python
labels = tf.ones((2, 1024), dtype=tf.int32)
input_ids = tf.ones((2, 1024), dtype=tf.int32)
print(model(input_ids, labels=labels, return_dict=True, training=True).loss.shape)
```
Outputs
```
TensorShape([2046])
```
## Expected behavior

According to the docs (https://huggingface.co/transformers/model_doc/gpt2.html#tfgpt2lmheadmodel), the loss is of shape `(1,)`. However, this is not the shape that is returned.
Hello !

This is an error in the documentation. TF doesn't apply a mean across all the values so you basically get a loss of shape 1023 (sequence length - 1 because of the right shift). Thanks for having spotted this!<|||||>This issue has been stale for 1 month.