Parallelize LongT5
### Feature request

Similar to regular T5, it'd be nice if LongT5 had parallelization support

### Motivation

This allows for training larger models / input sizes by using several gpus

### Your contribution

I have a pull request ready for this
Hi,

We've just deprecated the parallelize API as it can now be done using the `from_pretrained` method.

See https://github.com/huggingface/transformers/pull/21448