Decision Transformer Position Embedding Incorrect Implementation
### System Info

Not necessary (source code issue)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

From the code for decision transformer: https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L821-L822
But the actual implementation did not remove the position embedding from the GPT2 model
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L497
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L610-L611


### Expected behavior

The expected implementation should be 
https://github.com/kzl/decision-transformer/blob/e2d82e68f330c00f763507b3b01d774740bee53f/gym/decision_transformer/models/trajectory_gpt2.py#L680-L681
from the official decision transformer repo
cc @edbeeching @simoninithomas <|||||>Thanks for highlighting this, we set the position_ids to all zeros in the forward pass of the Decision Transformer Model: 
https://github.com/huggingface/transformers/blob/9fe2403bc52e342022ed132561655f84a6b6b7f3/src/transformers/models/decision_transformer/modeling_decision_transformer.py#L935-L942

In additional, the weights of this layer are loaded with zeros.  This is equivalent to not using the position embeddings. 
I just went through the Decision Transformer models on the hub to ensure that the model.encoder.wpe weights are indeed zeros and that is the case.

We left position embeddings in the implementation in case researchers wish to experiment with the inclusion of position embeddings. Please let us know if you find any other examples of potential bugs or require further clarification.