Fine-Tuning TFGPT2LMHeadModel / What to pass to fit
I am trying to fine-tune this pretrained model on my own data and I can't seem to get the format correct for what the model would like to see as input.

I am using TFGPT2LMHeadModel, GPT2Config and GPT2TokenizerFast.

When I do `model.fit(x,y, epochs=EPOCHS)`,

- If x and y are the outputs of tokenizing on GPT2TokenizerFast (i.e.  `tokenized = tokenizer(data_list, return_tensors='tf', add_special_tokens = True, truncation=True, padding = 'longest')`), I get: `ValueError: Unsupported value type BatchEncoding returned by IteratorSpec, serialize`.  I tried this because of what I saw on [this example code snippet](http://huggingface.co/transformers/model_doc/gpt2.html#tfgpt2lmheadmodel)
- If instead I choose x as `np.asarray(df['input_ids'].tolist()).astype('int32')` (and y is the corresponding thing for my label data) I get `InvalidArgumentError: Incompatible shapes: [32,154] vs. [2,32,16,154]` which looks a lot closer.

It seems like I have to choose the correct portions of the tokenizer output to feed to the fit function, but I am not choosing correctly.  Could you please clarify this for me?

I am using tensorflow 2.4.1 and transformers 4.5.1.
When I run code supplied by another user from [another issue](https://github.com/huggingface/transformers/issues/2439), which supposedly worked at one point in time, I get a similar dimension mismatch.  Is there a golden combination of tf and transformers I am supposed to be using?<|||||>Ah, the 16 above is the batch size, which must be 16.  If I create a dataset with this batch size, then my model with train with the fit function.  Like

`dataset = tf.data.Dataset.from_tensor_slices((x, y)) ` 
`dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)`<|||||>Hi, Tensorflow maintainer here! Can you paste me a minimal example that reproduces the problem? You don't have to share your data or anything, just give me a few lines with a made-up tiny dataset that I can run here to recreate the problem - it'll make it much easier for me to track it down. Alternatively, if you're loading your data from HF datasets, that's fine too.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.