Positinal Encoding for T5 family of models
### Feature request

Please create a hook to allow users to modify the T5 family of models and change the default positional embeddings to custom positional embeddings.

### Motivation

I am trying to build a T5 version that takes non-text input, and the traditional positional encodings are getting in the way, and there is no way to switch it off, or to make them learnable parameters, etc.
BART gives limited access to positional encodings, but T5 family gives nearly 0 access.

### Your contribution

If i knew where the positional encoding were calculated and added in to the input_ids, I could create this hook myself
Hi @SreehariSankar, thanks for raising this issue. 

Questions on designing custom hook for modifying the models are better placed in the [forum](https://discuss.huggingface.co/). 

All of the code for the model, including producing the embeddings are in the modeling files e.g. [this one for T5](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py). Note: the T5 family of models do not use the same positional embedding logic as in the traditional transformer i.e. there isn't a fixed embedding for each position, but instead a relative position embedding. 