bert.embeddings.position_ids is not loaded in TFBertForSequenceClassification
## Environment info

- `transformers` version: 4.17.0
- Python version: 3.7.11
- PyTorch version (GPU?): 1.10.1 (True)
- Tensorflow version (GPU?): 2.8.0 (False)

### Who can help

@LysandreJik

## Information

`TFBertForSequenceClassification` will not load `bert.embeddings.position_ids` from pytorch checkpoint, which will raise a warning.

**Related issue**: https://github.com/huggingface/transformers/issues/7797

## To reproduce

```python
from transformers import BertForSequenceClassification, TFBertForSequenceClassification

pt_model = BertForSequenceClassification.from_pretrained("uer/chinese_roberta_L-4_H-512")
pt_model.save_pretrained("pt_model")

# load pytorch model
tf_model = TFBertForSequenceClassification.from_pretrained("pt_model", from_pt=True)

# Expect: No warning/error
# Actual: This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
```

```
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']
- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.
```

Hey @z-bookworm 👋  From the name of the folder on the hub, it seems like you are trying to load a Roberta model ("uer/chinese_roberta_L-4_H-512") in a BERT model class. Can you try replacing `TFBertForSequenceClassification` with `TFRobertaForSequenceClassification`?

Let us know if it solves your problem :)<|||||>Hi @gante 👋

The model name may be a little bit misleading. According to the config.json and [README](https://huggingface.co/uer/chinese_roberta_L-4_H-512), it should use BERT model class indeed.

Trying to load the checkpoint with `RobertaForSequenceClassification` will raise a lot of warnings:

```
>>> pt_model = RobertaForSequenceClassification.from_pretrained("uer/chinese_roberta_L-4_H-512", local_files_only=True)
You are using a model of type bert to instantiate a model of type roberta. This is not supported for all configurations of models and can yield errors.
Some weights of the model checkpoint at uer/chinese_roberta_L-4_H-512 were not used when initializing RobertaForSequenceClassification: ['bert.embeddings.token_type_embeddings.weight', 'bert.encoder.layer.2.attention.output.dense.weight', 'bert.encoder.layer.0.output.LayerNorm.weight', 'bert.encoder.layer.3.attention.self.value.weight', 'bert.encoder.layer.1.attention.self.query.bias', 'bert.encoder.layer.0.attention.self.query.bias', 'bert.encoder.layer.1.output.dense.weight', 'bert.encoder.layer.1.attention.output.dense.weight', 'bert.encoder.layer.3.intermediate.dense.weight', 'bert.encoder.layer.2.attention.self.key.bias', 'bert.encoder.layer.3.intermediate.dense.bias', 'bert.encoder.layer.3.attention.self.key.weight', 'cls.predictions.decoder.bias', 'bert.encoder.layer.1.output.dense.bias', 'bert.encoder.layer.2.attention.self.value.weight', 'bert.encoder.layer.2.output.LayerNorm.bias', 'bert.encoder.layer.2.intermediate.dense.weight', 'bert.encoder.layer.2.attention.self.value.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias', 'bert.encoder.layer.0.attention.self.value.weight', 'bert.encoder.layer.1.attention.self.key.bias', 'cls.predictions.transform.LayerNorm.weight', 'bert.encoder.layer.2.attention.self.query.bias', 'bert.encoder.layer.0.attention.output.LayerNorm.bias', 'bert.encoder.layer.2.output.LayerNorm.weight', 'bert.encoder.layer.0.output.dense.bias', 'bert.encoder.layer.2.attention.output.LayerNorm.weight', 'bert.encoder.layer.0.output.dense.weight', 'bert.encoder.layer.3.output.dense.bias', 'bert.encoder.layer.3.output.LayerNorm.weight', 'bert.encoder.layer.0.intermediate.dense.weight', 'bert.encoder.layer.1.attention.self.query.weight', 'bert.encoder.layer.2.output.dense.weight', 'bert.encoder.layer.2.intermediate.dense.bias', 'bert.embeddings.LayerNorm.weight', 'bert.encoder.layer.1.output.LayerNorm.weight', 'bert.encoder.layer.0.attention.output.dense.bias', 'bert.encoder.layer.0.attention.output.LayerNorm.weight', 'bert.encoder.layer.1.attention.self.value.bias', 'bert.encoder.layer.3.attention.output.LayerNorm.weight', 'bert.encoder.layer.3.attention.output.LayerNorm.bias', 'bert.encoder.layer.1.intermediate.dense.weight', 'bert.encoder.layer.1.attention.self.key.weight', 'bert.embeddings.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.query.bias', 'bert.encoder.layer.3.attention.self.query.weight', 'bert.encoder.layer.3.attention.output.dense.weight', 'bert.encoder.layer.2.attention.self.key.weight', 'bert.encoder.layer.1.attention.output.LayerNorm.weight', 'bert.encoder.layer.0.attention.self.key.bias', 'bert.encoder.layer.0.attention.self.query.weight', 'bert.encoder.layer.0.intermediate.dense.bias', 'bert.encoder.layer.2.output.dense.bias', 'bert.encoder.layer.1.attention.output.dense.bias', 'bert.encoder.layer.2.attention.output.dense.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'bert.encoder.layer.1.attention.self.value.weight', 'bert.encoder.layer.0.attention.output.dense.weight', 'bert.embeddings.position_ids', 'bert.encoder.layer.3.output.dense.weight', 'bert.encoder.layer.0.output.LayerNorm.bias', 'bert.encoder.layer.2.attention.output.LayerNorm.bias', 'cls.predictions.bias', 'bert.encoder.layer.3.attention.output.dense.bias', 'bert.encoder.layer.0.attention.self.value.bias', 'bert.encoder.layer.1.intermediate.dense.bias', 'bert.encoder.layer.3.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.value.bias', 'bert.encoder.layer.1.output.LayerNorm.bias', 'bert.embeddings.position_embeddings.weight', 'bert.encoder.layer.2.attention.self.query.weight', 'bert.embeddings.word_embeddings.weight', 'bert.encoder.layer.0.attention.self.key.weight', 'bert.encoder.layer.1.attention.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.key.bias']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at uer/chinese_roberta_L-4_H-512 and are newly initialized: ['encoder.layer.0.output.dense.bias', 'encoder.layer.3.attention.output.LayerNorm.weight', 'encoder.layer.2.output.LayerNorm.weight', 'encoder.layer.3.attention.self.key.weight', 'encoder.layer.2.attention.self.key.weight', 'encoder.layer.0.attention.self.value.bias', 'encoder.layer.1.attention.output.dense.bias', 'encoder.layer.2.attention.self.value.bias', 'embeddings.LayerNorm.weight', 'encoder.layer.1.intermediate.dense.weight', 'encoder.layer.3.attention.self.key.bias', 'encoder.layer.3.attention.output.dense.weight', 'encoder.layer.3.attention.output.LayerNorm.bias', 'embeddings.token_type_embeddings.weight', 'encoder.layer.1.output.LayerNorm.weight', 'classifier.dense.bias', 'encoder.layer.2.intermediate.dense.weight', 'encoder.layer.0.attention.output.dense.weight', 'encoder.layer.3.output.LayerNorm.weight', 'encoder.layer.0.attention.self.query.bias', 'encoder.layer.1.output.dense.bias', 'encoder.layer.2.attention.output.dense.bias', 'encoder.layer.0.attention.self.key.bias', 'encoder.layer.0.intermediate.dense.weight', 'encoder.layer.0.attention.output.LayerNorm.bias', 'encoder.layer.0.output.LayerNorm.bias', 'encoder.layer.1.intermediate.dense.bias', 'encoder.layer.2.attention.self.query.weight', 'encoder.layer.2.attention.output.LayerNorm.bias', 'encoder.layer.1.attention.self.key.weight', 'encoder.layer.3.attention.self.value.weight', 'encoder.layer.2.attention.output.dense.weight', 'encoder.layer.3.attention.self.query.bias', 'encoder.layer.0.output.LayerNorm.weight', 'encoder.layer.1.attention.self.query.weight', 'encoder.layer.0.attention.self.query.weight', 'encoder.layer.3.intermediate.dense.bias', 'encoder.layer.1.attention.output.dense.weight', 'encoder.layer.0.attention.output.LayerNorm.weight', 'encoder.layer.2.attention.self.key.bias', 'encoder.layer.0.attention.output.dense.bias', 'embeddings.LayerNorm.bias', 'encoder.layer.1.attention.self.value.weight', 'encoder.layer.1.output.LayerNorm.bias', 'classifier.out_proj.weight', 'embeddings.word_embeddings.weight', 'encoder.layer.3.attention.output.dense.bias', 'encoder.layer.2.output.dense.bias', 'encoder.layer.2.output.dense.weight', 'encoder.layer.3.attention.self.value.bias', 'classifier.dense.weight', 'encoder.layer.2.output.LayerNorm.bias', 'classifier.out_proj.bias', 'embeddings.position_embeddings.weight', 'encoder.layer.2.attention.self.value.weight', 'encoder.layer.0.intermediate.dense.bias', 'encoder.layer.1.attention.self.query.bias', 'encoder.layer.1.attention.output.LayerNorm.weight', 'encoder.layer.0.output.dense.weight', 'encoder.layer.0.attention.self.value.weight', 'encoder.layer.2.intermediate.dense.bias', 'encoder.layer.3.attention.self.query.weight', 'encoder.layer.2.attention.output.LayerNorm.weight', 'encoder.layer.3.output.LayerNorm.bias', 'encoder.layer.1.attention.output.LayerNorm.bias', 'encoder.layer.3.output.dense.weight', 'encoder.layer.3.intermediate.dense.weight', 'encoder.layer.1.output.dense.weight', 'encoder.layer.2.attention.self.query.bias', 'encoder.layer.1.attention.self.key.bias', 'encoder.layer.1.attention.self.value.bias', 'encoder.layer.0.attention.self.key.weight', 'encoder.layer.3.output.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

According to the [source code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/roberta/modeling_tf_roberta.py#L117), it seems `position_ids ` is calculated deterministically without any parameters. It won't affect model performance. So can we remove this warning?<|||||>That's interesting! 🤔 

Because it is a non-expected combination of code/data (having a roberta model that should be loaded with bert, according to the instructions), that can happen. The only way to confirm that is correct is to do some standard checks, like confirming that the PT model has good performance on the expected downstream task, and that the TF model as the same output as the PT model for the same input. Because it is a model submitted by an user, I don't think there is much more we can do, and you should try to get in touch with the user if there is any remaining problem.

Since this is not a bug in the code, I'm closing this issue. For further (non-bug) discussions, please refer to our [forums](https://discuss.huggingface.co/) :) <|||||>@gante Sorry for using a misleading checkpoint as the example. **The problem should arise for any BERT model**. Take the official checkpoint `bert-base-uncased` for exmaple:

```python
from transformers import BertForSequenceClassification, TFBertForSequenceClassification

pt_model = BertForSequenceClassification.from_pretrained("bert-base-uncased")
pt_model.save_pretrained("pt_model")

tf_model = TFBertForSequenceClassification.from_pretrained("pt_model", from_pt=True)
```

```
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']
- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.
```<|||||>@z-bookworm that is a good point, I'm assigning the issue to me :)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>@gante Any progress on this?

I also had this issue when converting a bunch of models. It looks like the `position_ids` buffer is not used for anything anymore in the `Embeddings` class, and is just taking up memory.

```
from transformers import RobertaForMaskedLM, TFRobertaForMaskedLM

pt_model = RobertaForMaskedLM.from_pretrained('roberta-base')
pt_model.save_pretrained('./testing')
tf_model = TFRobertaForMaskedLM.from_pretrained('./testing', from_pt=True)
```

produces

```
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFRobertaForMaskedLM: ['roberta.embeddings.position_ids']
- This IS expected if you are initializing TFRobertaForMaskedLM from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFRobertaForMaskedLM from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFRobertaForMaskedLM were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFRobertaForMaskedLM for predictions without further training.
```<|||||>Hey @AndreasMadsen 👋  No update -- since it is not breaking, just an annoying warning that shouldn't be there, it's low in my priority.

If you'd like to have a go in fixing it, I'd be happy to provide pointers :)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>You can set the strict argument to False in the load_state_dict() function to ignore non-matching keys.
`model.load_state_dict(torch.load(path, map_location=torch.device('cpu')), strict=False)`