Converting tf weights:  AttributeError: 'GPT2Model' object has no attribute 'zeLoss'
# 🐛 Bug

## Information

Model I am using: gpt-2
I am trying to convert a fine-tuned gpt-2 model on tensroflow to pytorch state_dict. I used the nice script [here](https://github.com/huggingface/transformers/blob/ce50305e5b8c8748b81b0c8f5539a337b6a995b9/src/transformers/convert_gpt2_original_tf_checkpoint_to_pytorch.py).


## To reproduce

Steps to reproduce the behavior:

1. I run the following command:
```
python convert_gpt2_original_tf_checkpoint_to_pytorch.py --gpt2_checkpoint_path /home/finetune/model_best.ckpt --pytorch_dump_folder_path /home/gpt2-fine-tf2torch/
```
after running this command, I see the logs showing that the tf weights are being loaded but suddently I got the following error:

```
INFO:transformers.modeling_gpt2:Loading TF weight transformer_decoder/layer_9/self_attention/multihead_attention/value/bias with shape [1024]
INFO:transformers.modeling_gpt2:Loading TF weight transformer_decoder/layer_9/self_attention/multihead_attention/value/kernel with shape [1024, 1024]
INFO:transformers.modeling_gpt2:Loading TF weight word_embedder/w with shape [50257, 1024]
INFO:transformers.modeling_gpt2:Loading TF weight word_embedder_1/w with shape [50257, 1024]
Traceback (most recent call last):
  File "convert_gpt2_original_tf_checkpoint_to_pytorch.py", line 67, in <module>
    convert_gpt2_checkpoint_to_pytorch(args.gpt2_checkpoint_path, args.gpt2_config_file, args.pytorch_dump_folder_path)
  File "convert_gpt2_original_tf_checkpoint_to_pytorch.py", line 38, in convert_gpt2_checkpoint_to_pytorch
    load_tf_weights_in_gpt2(model, config, gpt2_checkpoint_path)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_gpt2.py", line 84, in load_tf_weights_in_gpt2
    pointer = getattr(pointer, scope_names[0])
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 585, in __getattr__
    type(self).__name__, name))
AttributeError: 'GPT2Model' object has no attribute 'zeLoss'
```

I tried to load the tf checkpoint and inspect variables, there were no 'zeLoss' but ''OptimizeLoss' which I guess the script mistakenly matched only the 'zeLoss':

```
>>> import tensorflow as tf
>>> import os
>>> tf_path = os.path.abspath('model_best.ckpt')
>>> tf_vars = tf.train.list_variables(tf_path)
```
here is part of the ```tf_vars```:
```
[('OptimizeLoss/beta1_power', []), ('OptimizeLoss/beta2_power', []), ('OptimizeLoss/position_embedder/w/Adam', [1024, 1024]), ('OptimizeLoss/position_embedder/w/Adam_1', [1024, 1024]), ('OptimizeLoss/transformer_decoder/beta/Adam', [1024]), ('OptimizeLoss/transformer_decoder/beta/Adam_1', [1024]), ('OptimizeLoss/transformer_decoder/gamma/Adam', [1024]), ('OptimizeLoss/transformer_decoder/gamma/Adam_1', [1024]), ('OptimizeLoss/transformer_decoder/layer_0/beta/Adam', [1024]), ('OptimizeLoss/transformer_decoder/layer_0/beta/Adam_1', [1024]), ('OptimizeLoss/transformer_decoder/layer_0/ffn/conv1/bias/Adam', [4096]), ('OptimizeLoss/transformer_decoder/layer_0/ffn/conv1/bias/Adam_1', [4096]), ('OptimizeLoss/transformer_decoder/layer_0/ffn/conv1/kernel/Adam', [1024, 4096]),
```

I would appreciate if you can help me fix this .
Thanks

@LysandreJik Any update on this? Thanks<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
