DeBERTa checkpoints contain `config` keys
We noticed some issues when loading PyTorch checkpoints for `DeBERTa`; the `state_dict` contained a `config` key with the config settings for the model, and this confused the cross-loading method. I don't think PyTorch state dicts are usually supposed to contain those. If possible, could we upload some new checkpoints that don't contain the configs?

@BigBird01 
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>@BigBird01 pinging on this one again!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.