DeBERTa xlarge v2 throwing runtime error
- `transformers` version: 4.4.2
- Platform: Linux-3.10.0-1127.el7.x86_64-x86_64-with-redhat-7.8-Maipo
- Python version: 3.6.13
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script: yes

```
RuntimeError: Error(s) in loading state_dict for DebertaForSequenceClassification:
        size mismatch for deberta.encoder.rel_embeddings.weight: copying a param with shape torch.Size([512, 1536]) from checkpoint, the shape in current model is torch.Size([1024, 1536]).
```

I've seen a previous post made about this error and i believe it is a known issue. On the thread i found it was mentioned that a fix was due to come out a month ago. Has the fix come out?

[Downloaded from here](https://huggingface.co/microsoft/deberta-v2-xlarge)
@roshan-k-patel this model has been trained using DebertaV2 and not Deberta, hence the size mismatch. Use DebertaV2ForSequenceClassification instead of DebertaForSequenceClassification 😃 .<|||||>Hi @frankhart2018, do you mean the [mnli variant](https://huggingface.co/microsoft/deberta-xlarge-mnli)?<|||||>No, @roshan-k-patel, I am referring to the model that you mentioned in the issue `deberta-v2-xlarge`. This is trained on DebertaV2 instead of Deberta architecture and hence requires instantiating `DebertaV2ForSequenceClassification`.<|||||>I see @frankhart2018, i used deberta base and large to for my task and it worked fine. For the x-large one i pulled the files from hugging face and scp to my server. I'm using the simpletransformers library to initialise and load the model. Are you familiar with the library and know how i might go about instantiating `DebertaV2ForSequenceClassification` instead? Also do you know the performance differences between the normal variants and mnli variants? Do the mnli ones tend to perform better on tasks. 

From the github repo [here](https://github.com/microsoft/DeBERTa), on the benchmarks it shows scores like **87.5/88.4**. Do you know if the second score after the '/' is the mnli version score? Thanks<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.