Training the TFGPT2LMHeadModel with model.fit produces error
MWE:
```python
from transformers.modeling_tf_gpt2 import TFGPT2LMHeadModel
from transformers.configuration_gpt2 import GPT2Config
from tensorflow.data import Dataset
import tensorflow as tf

data = tf.random.uniform(shape=[10000], dtype = tf.int32, maxval = 100)
src = tf.constant(data)
def split_input_target(chunk):
    return chunk[:-1], chunk[1:]
ds = Dataset.from_tensor_slices(src) \
            .batch(256 + 1, drop_remainder = True) \
            .map(split_input_target)
model = TFGPT2LMHeadModel(GPT2Config())
loss = ['sparse_categorical_crossentropy'] + [None] * 12
model.compile(loss = loss, metrics = ['sparse_categorical_accuracy'])
model.fit(ds)
```
This generates the error `ValueError: Dimensions must be equal, but are 256 and 12 for '{{node Equal_1}} = Equal[T=DT_FLOAT, incompatible_shape_error=true](Cast_6, Cast_7)' with input shapes: [256,1], [2,256,12,1].` Is there any explanation somewhere on how to train like this?




Hello!

Your way of computing the loss is wrong. I suggest you to look at how we compute it [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_gpt2.py#L650) and [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_utils.py#L125). You should also rewrite your metric the same way.<|||||>I know but changing the loss metric won't fix the ValueError.<|||||>It is. Just look at how we do it. Including in the tests.<|||||>Met the same error before, it works for me with removing the `metrics` params when `model.compile`. But I think it's a bad idea.<|||||>Hi, there. I found it won't raise this error if `batch size` == `n_head`. For example, LysandreJik's [gits](https://gist.github.com/LysandreJik/c958925768eb6a9a72609ea99561d1cb)  works only `BATCH_SIZE = 12`<|||||>~~Hey, guys. If the output of other layers was used rarely, can we add a controller to select whether return multi-layer logits or not? (correct me if it's necessary to return multi-layer logits.)~~

~~If we can do this, this [pull request](https://github.com/huggingface/transformers/pull/8584) may help.~~<|||||>@bjourne  Hey guy, I think adding `output_attentions=False`  and `output_hidden_states=False` may help:
```
model = TFGPT2LMHeadModel(GPT2Config(output_attentions=False, output_hidden_states=False, use_cache=False))
```<|||||>I haven't had the chance to try that. Maybe someone else can? Will the trained network with `output_hidden_states=True` though? You need to set it to True when generating text.<|||||>Hello,

I got the same error.
When I tried to set `output_attentions = False`, it didn't seem to be in GPT2Config.
Looking at the [documentation](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2config), GPT2Config doesn't have that parameter, where should I set it?<|||||>> Hello,
> 
> I got the same error.
> When I tried to set `output_attentions = False`, it didn't seem to be in GPT2Config.
> Looking at the [documentation](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2config), GPT2Config doesn't have that parameter, where should I set it?

`output_attentions` is a Parameter of `PretrainedConfig` which is the SuperClass of `GPT2Config`, [see](https://huggingface.co/transformers/main_classes/configuration.html#transformers.PretrainedConfig) <|||||>Thank you for your reply.

> PretrainedConfig which is the SuperClass of GPT2Config

I see, I understand.
I noticed that the version of the transformer I'm using was out of date :cry: 

> GPT2Config(output_attentions=False, output_hidden_states=False, use_cache=False)

I used the latest version and it worked without any errors!
Your advice was very helpful, thank you!<|||||>This issue has been stale for 1 month.