absolute embeddings in Deberta
The paper says they add absolute position embeddings at the last layer, however, the models is still using the addition of position embeddings and word embeddings.


In configuration_deberta.py
------------
 position_biased_input (:obj:`bool`, `optional`, defaults to :obj:`True`):
          Whether add absolute position embedding to content embedding.

default setting: position_biased_input=True 



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>> The paper says they add absolute position embeddings at the last layer, however, the models is still using the addition of position embeddings and word embeddings.
> 
> ## In configuration_deberta.py
> position_biased_input (:obj:`bool`, `optional`, defaults to :obj:`True`):
> Whether add absolute position embedding to content embedding.
> 
> default setting: position_biased_input=True

It's disabled by the model_config.json along with the model repository.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>The implementation of Deberta is actually not the same as the paper, the absolute embeddings should be added at the last two layers.