Error while loading model file - .ckpt file :: Missing key(s) in state_dict
@julien-c @patrickvonplaten @thomwolf 
transformers version == 4.0.0

I trained a T5 model for classification and I am trying to load the checkpoint model saved after 3 epochs. 
While trying to load the model, getting the below error.
![image](https://user-images.githubusercontent.com/54103522/102682082-a0281200-41ec-11eb-8292-4fbeba807ec3.png)


The model file is a ckpt file.

Any help is appreciated.
Thanks.

Hi @adithyaan-creator 

Seems like you used `pytorch-lightning` for training, `pl` prefixes every `state_dict` key with `model.` so to load the `pl` checkpoint to hf format, remove the `model.` prefix from `state_dict` keys.

```python3
def remove_prefix(text: str, prefix: str):
    if text.startswith(prefix):
        return text[len(prefix) :]
    return text  # or whatever

state_dict = {remove_prefix(k, "model."): v for k, v in state_dict.items()}
hf_model.load_state_dict(state_dict)
hf_model.save_pretrained(save_path)
```
Also, would be nice if you could post code snippet instead of screenshots :)

<|||||>Hi @patil-suraj,
I tried it with loading the 'state_dict' item to the model, and getting the following error "Unexpected key(s) in state_dict: "decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight". ".

The code is :: 
def remove_prefix(text: str, prefix: str):
  if text.startswith(prefix):
        return text[len(prefix) :]
    return text  # or whatever

ckpt = {remove_prefix(k, "model."): v for k, v in ckpt['state_dict'].items()}

model.load_state_dict(ckpt)
model.save_pretrained("/content/model")`
  <|||||>Hey @adithyaan-creator,

You can ignore this warning, see: https://github.com/huggingface/transformers/pull/9231<|||||>@patrickvonplaten the code that was merged wasnt working out, but since the weight was unnecessary I deleted the weight layer and it worked out.
'del ckpt["decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight"]'
Thanks.<|||||>@adithyaan-creator Hi, I am having the same issue now. Could you elaborate on how and where did you add 'del ckpt["decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight"]' ? Thank you very much!