Add Switch transformers
# What does this PR do?

This PR attempts to add Switch Transformers from t5x with @ArthurZucker & @thomwolf 

The architecture seems to be similar to a t5 architecture (the architecture is copied from T5), where the FF layer is slightly modified, introducing the first Mixture of Experts (MoE) architecture inside `transformers` library.

paper: https://arxiv.org/abs/2101.03961
weights: https://github.com/google-research/t5x/blob/eb42c2524bf65c8a46624f1a9b9e034d9bc65b14/docs/models.md#converted-mesh-tensorflow-checkpoints
original modeling code: https://github.com/google/flaxformer/tree/b725bd2a51d70e866d819c92de166fbf24425e6a/flaxformer/architectures/moe


# TODOs:

- [x] Make the forward pass run
- [x] Convert the weights in Pytorch format and upload them on the Hub
- [x] Match the logits between the original implementation and ours
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.<|||||>Thanks a lot @sgugger for your comments! 
Would love to have another round of review as I added some modification for `accelerate` and `bnb` compatibility 🙏 <|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19323). All of your documentation changes will be reflected on that endpoint.<|||||>Failing tests seems to be unrelated to this PR, merging!