[Some weights or buffers of the PyTorch model TFGPT2LMHeadModel were not initialized] convert GPT2 pytorch to tensorflow model 
# ❓ Questions & Help

How can we convert gpt2 fine tuned model from run_language_model.py from pytorch to tensorflow model? I came into the following warning when importing pytorch model GPT2LMHeadModel into TFGPT2LMHeadModel.

```
WARNING:transformers.modeling_tf_pytorch_utils:Some weights or buffers of the PyTorch model TFGPT2LMHeadModel were not initialized from the TF 2.0 model and are newly initialized: ['transformer.h.2.attn.bias', 'transformer.h.4.attn.masked_bias', 'transformer.h.4.attn.bias', 'transformer.h.3.attn.bias', 'transformer.h.3.attn.masked_bias', 'transformer.h.0.attn.masked_bias', 'transformer.h.5.attn.masked_bias', 'transformer.h.1.attn.bias', 'transformer.h.0.attn.bias', 'transformer.h.5.attn.bias', 'transformer.h.1.attn.masked_bias', 'lm_head.weight', 'transformer.h.2.attn.masked_bias']
```

Or do we have any tutorial to train a GPT2 language model by TFTrainer? 

## Details
<!-- Description of your issue -->

Reproduce

```
from transformers import *
model = GPT2LMHeadModel.from_pretrained("distilgpt2")
model.save_pretrained("./gpt2_pt")
model = TFGPT2LMHeadModel.from_pretrained("./gpt2_pt", from_pt=True)
```

However we won't see the above warning if we use 
```
model = TFGPT2LMHeadModel.from_pretrained("distilgpt2")
```

If we already use `run_language_model.py` to train a GPT2LMHeadModel model and import it into TFGPT2LMHeadModel, can we safely use the converted model even if we see the warning?
Hi! The way you converted your model is the recommended way :ok_hand: 

The warning is irrelevant to this conversion, we should try to make that clearer.<|||||>The warning is very confusing and can be fixed by pull request #6604.<|||||>The warning is very confusing and can be fixed by new pull request #6623