Unexpected weights were not initialized from the model checkpoint error 
## Environment info

- `transformers` version: 4.9.2
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): 2.6.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no 

### Who can help
@LysandreJik @sgugger

## Information

I am defining a simple multi-class BERT classification model and then training it using pytorch-lightning. The code is below under class `BertForMulticlassSequenceClassification(BertPreTrainedModel)`. The issue is that after training when I am loading the classifier model `model = ClassTaggerModel.load_from_checkpoint(checkpoint_file)` I get 
```
Some weights of BertForMulticlassSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifiers.0.weight', 'classifiers.1.bias', 'classifiers.0.bias', 'classifiers.1.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```
This is quite strange as I am loading BertForMulticlassSequenceClassification checkpoint. In fact if I inspect state_dict with torch I can see `model.classifiers.0.weight` etc, so they are in there. Any suggestions on why I am still getting this error would be most appreciated! Thank you! 

## Reproduce 
Extract notebook https://colab.research.google.com/drive/1os9mz7w7gmLBL_ZDvZ9K1saz9UA3rmD7?usp=sharing and minimal data files attached here and "run all". 
[classes.txt](https://github.com/huggingface/transformers/files/7085157/classes.txt)
[train.csv](https://github.com/huggingface/transformers/files/7085158/train.csv)
[val.csv](https://github.com/huggingface/transformers/files/7085159/val.csv)

You are not providing the code necessary to reproduce the error so there is little we can do to help. Also note that code like `model = ClassTaggerModel.load_from_checkpoint(checkpoint_file)` does not execute anything from the Transformers library as it uses PyTorch Lightning, so it's unlikely you are getting the warning printed above with it.<|||||>@sgugger thanks for fast reply - I have added minimal example in my original report above - the link is to colab. It does a simple one epoch training on a subset of data (seconds) and saves checkpoint. Then it loads checkpoint leading to reported error. <|||||>I'm sorry but I'm really confused as to why you are opening an issue here: you are saving a checkpoint with PyTorch ligthning and reloading it with PyTorch lightning, not Transformers.