DeBerta Wrong Dimension for MLM Prediction Head
### System Info

From the [original implementation](https://github.com/microsoft/DeBERTa/blob/master/DeBERTa/deberta/mlm.py#L21), the MLM head transforms the hidden dimension to the embedding dimension. 

However, it seems that in the HF version, we go from `hidden_size` to `hidden_size`. Shouldn't it be from `hidden_size` to `embedding_size`, especially since the [embeddings get tied](https://github.com/huggingface/transformers/blob/v4.24.0/src/transformers/modeling_utils.py#L1203) eventually?

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

` DebertaPredictionHeadTransform.dense.weight` should be of size `hidden_size, embedding_size`, not `hidden_size, hidden_size`
Not sure I understand where you found that we only have from `hidden_size` to `hidden_size`, but for `MLM`, we use the `DebertaOnlyMLMHead`, which uses a `decoder` that is the prediction head. See `self.decoder = nn.Linear(config.hidden_size, config.vocab_size, bias=False)` [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/deberta/modeling_deberta.py#L1146)<|||||>Ah sorry I forgot to include this line!
https://github.com/huggingface/transformers/blob/main/src/transformers/models/deberta_v2/modeling_deberta_v2.py#L872
It looks like it checks for an attribute `embedding_size` in the config, but defaults to `hidden_size` if not found. 

It doesn’t seem like that attribute is present in the config though? Am I setting up the model incorrectly?<|||||>Okay, again the head that you mentioned : `DebertaPredictionHeadTransform` is just a head, and it does not use the `DebertaV2Embedding`. It is only used as such (and not as an entire model). The size is correct as what you are looking for it in the `DebertaForMaskedLM`. The `self.model` attribute encompasses the `embedding` layer. <|||||>Hm okay thanks for the help! Seems I misunderstood